SNOMED International Release Management

Page tree

Versions Compared

Old Version 2

changes.mady.by.user Andrew Atkinson

Saved on

compared with

New Version Current

changes.mady.by.user Andrew Atkinson

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

June 2023 PreProduction (Version 5 - Fixes from version 4) traceability vs June 2023

...

Production (Version

...

1) traceability

SUMMARY COMPONENT REPORT:  https://docs.google.com/spreadsheets/d/1Q18jq4eHU4-H8hou3U3Pb-4l1jGgqRNSIk-1KvTfrIo/edit#gid=1

MSSP TICKET:  https://jira.ihtsdotools.org/browse/MSSP-2072

Differences found in package Comparison

Number of RF2 records impactedRelated JIRA ticket(s)Rationale

...

2 records added, denoting 1 new refset as expected ????

> 1ca3e72a-71ee-436d-9c06-e47e0db53185 20220607 1 2011000195101 900000000000456007 22241000195100 900000000000510002 900000000000462002 0
> 1dda242a-587b-4b38-8eff-f86b509afa22 20220607 1 2011000195101 900000000000456007 22241000195100 900000000000511003 900000000000461009 1

Checking with Pero in MSSP-1518 that this language refset addition is expected...

...

3 records had their UUID's replaced, BUT ALSO HAD DUPLICATE RECORDS ADDED (presumably by the automated ModuleDependency code):

< 36ddb438-742a-4c97-9dca-b95489b6a08a 20211207 1 2011000195101 900000000000534007 900000000000207008 20211207 20210731
> 36ddb438-742a-4c97-9dca-b95489b6a08a 20220607 1 2011000195101 900000000000534007 900000000000207008 20220607 20220331
> 59cf1492-b48d-4423-bd8d-f690ce8f6e73 20211207 1 2011000195101 900000000000534007 900000000000207008 20211207 20210731

< 7321d59a-986d-4496-9cf8-874022470ad5 20211207 1 11000241103 900000000000534007 900000000000012004 20211207 20210731
> 7321d59a-986d-4496-9cf8-874022470ad5 20220607 1 11000241103 900000000000534007 900000000000012004 20220607 20220331
> 293406ae-3aef-4393-b588-f72479f3075e 20211207 1 11000241103 900000000000534007 900000000000012004 20211207 20210731

< a4fbf40a-73ae-4e51-80da-cddc448d3a32 20211207 1 2011000195101 900000000000534007 900000000000012004 20211207 20210731
> a4fbf40a-73ae-4e51-80da-cddc448d3a32 20220607 1 2011000195101 900000000000534007 900000000000012004 20220607 20220331
> 3e4b83c5-6e40-4793-8d8f-3bf6e0379958 20211207 1 2011000195101 900000000000534007 900000000000012004 20211207 20210731

Also 2 records had their UUID's replaced unnecessarily:

< b3c09a5c-496c-48d7-808a-166f2b6a3c97 20211207 1 11000241103 900000000000534007 900000000000207008 20211207 20210731
> 902510db-1db3-4635-88d0-956467801be5 20211207 1 11000241103 900000000000534007 900000000000207008 20211207 20210731

< dc968531-82da-4aeb-bf59-3ae96cf71dd7 20211207 1 2011000195101 900000000000534007 11000241103 20211207 20210930
> 1b57c717-740e-46c3-8ea1-1726d4c541e1 20211207 1 2011000195101 900000000000534007 11000241103 20211207 20210930

SO ALL WRONG!! NEED TO REVERT BACK TO EXTERNALLYMAINTAINED INPUT FOR THIS FILE TO SEE IF THIS FIXES IT!!

Readme filen/a

All x prefixes removed, and Production naming conventions applied, as expected


June 2023 PreProduction (Version 4 - COMPLETE RE-START OF SWISS RELEASE, AS IF THIS IS A "FIRST TIME" RELEASE) traceability

 vs June 2023 PreProduction (Version 5 - Fixes from version 4) traceability

SUMMARY COMPONENT REPORT:  https://docs.google.com/spreadsheets/d/1Q18jq4eHU4-H8hou3U3Pb-4l1jGgqRNSIk-1KvTfrIo/edit#gid=1

MSSP TICKET:  https://jira.ihtsdotools.org/browse/MSSP-2072

Differences found in package Comparison

Number of RF2 records impactedRelated JIRA ticket(s)Rationale
RefsetDescriptor files10

8 records had their UUID's reverted to December 2022 (plus 2 records that were missing from the PreProd version 4 package) + new effectiveTime of 20230607 (but NO CHANGES TO ACTUAL RECORDS) as expected, due to NEW "FIRST TIME" RELEASE:

  • < 548dc527-855a-476a-9e19-183666f89c8d    20230607    1    2011000195101    900000000000456007    2031000195108    900000000000511003    900000000000461009    1
    < c1f1334c-9bb0-4149-a2ea-00914853e3cf    20230607    1    2011000195101    900000000000456007    22241000195100    900000000000510002    900000000000462002    0
    < c6263863-9152-4e93-b229-d8b0243d8e6b    20230607    1    2011000195101    900000000000456007    2021000195106    900000000000511003    900000000000461009    1
    < d9196a6b-29a1-4dbd-bb6b-9346429d9dfc    20230607    1    2011000195101    900000000000456007    2041000195100    900000000000511003    900000000000461009    1
    < e654efc1-6113-4eff-acb9-a08ade12dc7e    20230607    1    2011000195101    900000000000456007    2041000195100    900000000000510002    900000000000462002    0
    < eb71ee49-88b2-4713-950a-4506e714b456    20230607    1    2011000195101    900000000000456007    2031000195108    900000000000510002    900000000000462002    0
    < eb9f152b-79f1-4d3f-8b8a-6381cb076d45    20230607    1    2011000195101    900000000000456007    22241000195100    900000000000511003    900000000000461009    1
    < fdf18dfc-59ac-4ef6-a844-9b883e9dc531    20230607    1    2011000195101    900000000000456007    2021000195106    900000000000510002    900000000000462002    0
  • > 174062a5-02eb-4179-899c-741faa68d9a6    20230607    1    2011000195101    900000000000456007    2031000195108    900000000000510002    900000000000462002    0
    > 1ca3e72a-71ee-436d-9c06-e47e0db53185    20230607    1    2011000195101    900000000000456007    22241000195100    900000000000510002    900000000000462002    0
    > 1dda242a-587b-4b38-8eff-f86b509afa22    20230607    1    2011000195101    900000000000456007    22241000195100    900000000000511003    900000000000461009    1
    > 2086b060-a11f-4409-867c-56c308f356ea    20230607    1    2011000195101    900000000000456007    2041000195100    900000000000511003    900000000000461009    1
    > 31a3a105-910b-4c7f-842d-1b0e7b9b89de    20230607    1    2011000195101    900000000000456007    2021000195106    900000000000510002    900000000000462002    0
    > 4ae80dc6-2654-4188-85c9-4e5003c96c6f    20230607    1    2011000195101    900000000000456007    2041000195100    900000000000510002    900000000000462002    0
    > 58b37409-0e3b-49ec-9cc4-f2ea835a5789    20230607    1    2011000195101    900000000000456007    722131000    900000000000510002    900000000000462002    0
    > 8afd2ce3-9943-4359-bf0a-18f812ae1bbd    20230607    1    2011000195101    900000000000456007    2031000195108    900000000000511003    900000000000461009    1
    > b9ca50ba-2a5e-4cda-806b-aa313a84dcf9    20230607    1    2011000195101    900000000000456007    722131000    900000000000511003    900000000000461009    1
    > ef7acdd0-afcd-416b-8968-358b8d5dfa88    20230607    1    2011000195101    900000000000456007    2021000195106    900000000000511003    900000000000461009    1

So when compared to the old December 2022 release, the new Version 5 refsetDescriptor records are identical except for the new effectiveTime of 20230607 for each record, as expected.

ModuleDependency files2

2 records added back in as expected, as remaining 3 from December 2022 were all related to the old dependency on the CF module, which no longer exists!

  • > 36ddb438-742a-4c97-9dca-b95489b6a08a    20230607    1    2011000195101    900000000000534007    900000000000207008    20230607    20230430
  • > a4fbf40a-73ae-4e51-80da-cddc448d3a32    20230607    1    2011000195101    900000000000534007    900000000000012004    20230607    20230430

Therefore, when compared to the old December 2022 release, the new Version 5 moduleDependency records are identical (except for the new effectiveTime of 20230607 for each record) for the 2 records that are still applicable in the new world of hosting all CF + CG records in the CH module instead of in the CF + CG modules.  However, we've now removed 3 of the old moduleDependency records, as we no longer need to call out the dependencies on those CF + CG modules:

  • b3c09a5c-496c-48d7-808a-166f2b6a3c97    20221207    1    11000241103    900000000000534007    900000000000207008    20221031    20221031

  • dc968531-82da-4aeb-bf59-3ae96cf71dd7    20221207    1    2011000195101    900000000000534007    11000241103    20221207    20221031

  • 7321d59a-986d-4496-9cf8-874022470ad5    20221207    1    11000241103    900000000000534007    900000000000012004    20221031    20221031

This is all now exactly as expected.

Description-fr FULL + SNAPSHOT files7478

7478 inactive CF module (11000241103) records have been completely removed from the package.

This is exactly as expected, due to the fixes that were applied by Rory after the remaining 7478 CF module records were identified in the Version 4 testing below...

Description-fr-ch FULL + SNAPSHOT files7478

7478 inactive records (from the -fr file, that were removed because they were still in the CF module (11000241103)) have now been added back into the package in the RIGHT MODULE (the Swiss module 2011000195101)

This is exactly as expected, due to the fixes that were applied by Rory after the remaining 7478 CF module records were identified in the Version 4 testing below...


December 2022 Production (PUBLISHED) vs June 2023 PreProduction (Version 4 - COMPLETE RE-START OF SWISS RELEASE, AS IF THIS IS A "FIRST TIME" RELEASE) traceability

SUMMARY COMPONENT REPORT:  https://docs.google.com/spreadsheets/d/1Q18jq4eHU4-H8hou3U3Pb-4l1jGgqRNSIk-1KvTfrIo/edit#gid=1

MSSP TICKET:  https://jira.ihtsdotools.org/browse/MSSP-2072

Differences found in package Comparison

Number of RF2 records impactedRelated JIRA ticket(s)Rationale
RefsetDescriptor files10

8 records had their UUID's refreshed + new effectiveTime of 20230607 (but NO CHANGES TO ACTUAL RECORDS) as expected, due to NEW "FIRST TIME" RELEASE:

  • Dec 22
    • < 174062a5-02eb-4179-899c-741faa68d9a6    20211207    1    2011000195101    900000000000456007    2031000195108    900000000000510002    900000000000462002    0
    • < 1ca3e72a-71ee-436d-9c06-e47e0db53185    20220607    1    2011000195101    900000000000456007    22241000195100    900000000000510002    900000000000462002    0
    • < 1dda242a-587b-4b38-8eff-f86b509afa22    20220607    1    2011000195101    900000000000456007    22241000195100    900000000000511003    900000000000461009    1
    • < 2086b060-a11f-4409-867c-56c308f356ea    20211207    1    2011000195101    900000000000456007    2041000195100    900000000000511003    900000000000461009    1
    • < 31a3a105-910b-4c7f-842d-1b0e7b9b89de    20211207    1    2011000195101    900000000000456007    2021000195106    900000000000510002    900000000000462002    0
    • < 4ae80dc6-2654-4188-85c9-4e5003c96c6f    20211207    1    2011000195101    900000000000456007    2041000195100    900000000000510002    900000000000462002   0
    • < 8afd2ce3-9943-4359-bf0a-18f812ae1bbd    20211207    1    2011000195101    900000000000456007    2031000195108    900000000000511003    900000000000461009    1
    • < ef7acdd0-afcd-416b-8968-358b8d5dfa88    20211207    1    2011000195101    900000000000456007    2021000195106    900000000000511003    900000000000461009    1
  • June 23
    • > 548dc527-855a-476a-9e19-183666f89c8d    20230607    1    2011000195101    900000000000456007    2031000195108    900000000000511003    900000000000461009    1
    • > c1f1334c-9bb0-4149-a2ea-00914853e3cf    20230607    1    2011000195101    900000000000456007    22241000195100    900000000000510002    900000000000462002    0
    • > c6263863-9152-4e93-b229-d8b0243d8e6b    20230607    1    2011000195101    900000000000456007    2021000195106    900000000000511003    900000000000461009    1
    • > d9196a6b-29a1-4dbd-bb6b-9346429d9dfc    20230607    1    2011000195101    900000000000456007    2041000195100    900000000000511003    900000000000461009    1
    • > e654efc1-6113-4eff-acb9-a08ade12dc7e    20230607    1    2011000195101    900000000000456007    2041000195100    900000000000510002    900000000000462002    0
    • > eb71ee49-88b2-4713-950a-4506e714b456    20230607    1    2011000195101    900000000000456007    2031000195108    900000000000510002    900000000000462002    0
    • > eb9f152b-79f1-4d3f-8b8a-6381cb076d45    20230607    1    2011000195101    900000000000456007    22241000195100    900000000000511003    900000000000461009    1
    • > fdf18dfc-59ac-4ef6-a844-9b883e9dc531    20230607    1    2011000195101    900000000000456007    2021000195106    900000000000510002    900000000000462002    0

2 records however, DID NOT GET REPLACED IN NEW JUNE 2023 RELEASE as expected???

  • < 58b37409-0e3b-49ec-9cc4-f2ea835a5789    20211207    1    2011000195101    900000000000456007    722131000    900000000000510002    900000000000462002    0
  • < b9ca50ba-2a5e-4cda-806b-aa313a84dcf9    20211207    1    2011000195101    900000000000456007    722131000    900000000000511003    900000000000461009    1


ModuleDependency files5

5 records removed completey, replaced with nothing!!

SO ALL WRONG!! NEED TO REVERT BACK TO EXTERNALLYMAINTAINED INPUT FOR THIS FILE TO SEE IF THIS FIXES IT!!

Readme file

all x prefixes added in, plus PreProduction naming convention applied, as expected. 

Association files12

2 new records added/updated for 20230607, 

10 records (from all different effectiveTimes) removed

...NOT as expected due to Summary Component Stats report for this cycle (see link above) showing:

  • 2 new records added/updated for 20230607, 

  • 0 records inactivated

IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

ALL HISTORY GONE FROM ASSOCIATION FILES - I GUESS THIS IS WHAT WE WERE AIMING FOR WITH A BRAND NEW RELEASE, BUT JUST NEED TO DOUBLE CHECK WITH RORY...

Yes, Rory confirmed this is correct during our call at 17:00 on  

AttributeValue files11613

6860 new records added/updated for 20230607, 

11613 records (from all different effectiveTimes) removed

...NOT as expected due to Summary Component Stats report for this cycle (see link above) showing:

  • 6860 new records added/updated for 20230607, 

  • 0 records inactivated

IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

ALL HISTORY GONE FROM ATTRIBUTEVALUE FILES - I GUESS THIS IS WHAT WE WERE AIMING FOR WITH A BRAND NEW RELEASE, BUT JUST NEED TO DOUBLE CHECK WITH RORY...

Yes, Rory confirmed this is correct during our call at 17:00 on  

Language Refset (de-ch) files29491

27928 new records added/updated for 20230607 - ***** FOR REFSETID: 2041000195100 *****

1563 records (from all different effectiveTimes) removed

..NOT as expected due to Summary Component Stats report for this cycle (see link above) showing:

  • 27928 new records added/updated for 20230607, 

  • 0 records inactivated

DOES THIS MEAN THE SCS REPORT WILL BE INVALID, AS WILL SHOW NOW INACTIVATIONS BECAUSE IT'S A "BRAND NEW RELEASE" WITH NO "PREVIOUSPUBLISHEDPACKAGE" TO COMPARE THE CONTENT TO????

So my manual validation shows:

  1. 99 inactive records (ALL OF THEM) were removed from the original Language refset file, as expected
  2. 1465 active records were removed from the original Language refset file, all in refset: 2041000195100 (Swiss German LRS)
    1. These were all in the Swiss Module (2011000195101), so not sure why we removed these, other than to just wipe the slate clean and start again??
    2. GOOD NEWS is that I spot checked several dozen and they're all JUST UUID REFRESHES, as they all exist in the new content being added back in, so all good as we'd expect the UUID's to be refreshed due to this being a "Brand new release"
  3. 27,928 records added back in!!!
    1. All in the Swiss Module (2011000195101) as expected
    2. So assuming that the 1465 original records were all replaced (just had UUID's refreshed), we're looking at 26,463 NEW records from the Common German import.
    3. PLUS 27,928 matches exactly to the SCS report above

IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

Yes, Rory confirmed this is correct during our call at 17:00 on  

Language Refset (it-ch) files2603

1292 new records added/updated for 20230607 - ***** FOR REFSETID: 2031000195108 *****

1311 records (from all different effectiveTimes) removed

So my manual validation shows:

  1. 32 inactive records (ALL OF THEM) were removed from the original Language refset file, as expected
  2. 1279 active records were removed from the original Language refset file, all in refset: 2031000195108 (Swiss Italian LRS)
    1. This accounts for the removal of ALL it-ch records from previous package, as expected as inactives removed forever, and actives removed just to refresh UUID's
  3. 1292 records added back in!! (all with 20230607 effectiveTime, all in SWISS module (2011000195101) + all in refset: 2031000195108 (Swiss Italian LRS)
    1. From dozens of spot checks this looks like 1279 original active records (just has UUID's refreshed) + 13 NEW records for this cycle
    2. PLUS 1292 matches exactly to the SCS report above

...as expected ????????????????

Yes, Rory confirmed this is correct during our call at 17:00 on  

Language Refset (en) files182

96 new records added/updated for 20230607 - ***** FOR REFSETID: 900000000000508004 + 900000000000509007 *****

86 records (from all different effectiveTimes) removed

So my manual validation shows:

  1. 0 inactive records (ALL OF THEM) were removed from the original Language refset file, as expected
  2. 86 active records were removed from the original Language refset file, all in refset: 900000000000509007 (en-US LRS) + 900000000000508004 (en-GB LRS)
    1. This accounts for the removal of ALL it-ch records from previous package, as expected as inactives removed forever, and actives removed just to refresh UUID's
  3. 98 records added back in!! (all with 20230607 effectiveTime, all in SWISS module (2011000195101) + all in refset: 900000000000509007 (en-US LRS) + 900000000000508004 (en-GB LRS)
    1. From dozens of spot checks this looks like 86 original active records (just has UUID's refreshed) + 12 NEW records for this cycle
    2. PLUS 98 matches exactly to the SCS report above

IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

Yes, Rory confirmed this is correct during our call at 17:00 on  

Language Refset (fr) files0

0 records added/updated, as expected as this is Common French Language file??????

SUMMARY REPORT EXPECTING 0

Language Refset (fr-ch) files260,288

161,454 new records added/updated for 20230607 - ***** FOR REFSETID: 2021000195106 *****

98,834 records (from all different effectiveTimes) removed

So my manual validation shows:

  1. 6059 inactive records (ALL OF THEM) were removed from the original Language refset file, as expected
  2. 92,775 active records were removed from the original Language refset file, all in refset: 2021000195106 (Swiss French LRS)
    1. These were all in the Swiss Module (2011000195101), so not sure why we removed these, other than to just wipe the slate clean and start again??
      1. ******* BUT I THOUGHT WE WERE DOING SOMETHING DIFFERENT IN THIS CYCLE WITH TEH COMMON FRENCH APPROACH??  MAYBE JUST IN DESCRIPTIONS??????? *******
    2. GOOD NEWS is that I spot checked several dozen and they're all JUST UUID REFRESHES, as they all exist in the new content being added back in, so all good as we'd expect the UUID's to be refreshed due to this being a "Brand new release"
  3. 161,454 records added back in!!! (all with 20230607 effectiveTime, all in SWISS module (2011000195101) + all in refset: 2021000195106 (Swiss French LRS)
    1. All in the Swiss Module (2011000195101) as expected
    2. From dozens of spot checks it looks like the 92,775 original records were all replaced (just had UUID's refreshed), so we're looking at 68,679 NEW records from the Common French import.
    3. PLUS 161,454 matches exactly to the SCS report above

IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

Yes, Rory confirmed this is correct during our call at 17:00 on  

Concept files46

25 new records added/updated for 20230607, 

21 records (from all different effectiveTimes) removed

...NOT as expected due to Summary Component Stats report for this cycle (see link above) showing:

  • 25 new records added/updated for 20230607, 

  • 0 records inactivated

DOES THIS MEAN THE SCS REPORT WILL BE INVALID, AS WILL SHOW NOW INACTIVATIONS BECAUSE IT'S A "BRAND NEW RELEASE" WITH NO "PREVIOUSPUBLISHEDPACKAGE" TO COMPARE THE CONTENT TO????

Some of the records removed appear to be absolutely valid, as they are previously inactive records (and therefore there's no need to keep them as "born inactive" records) (eg)

  • 394812008    20220607    0    2011000195101    900000000000074008

Many more appear to be valid as they're simply "removing" these records to be replaced with the same record just with new effectiveTimes (eg)

  • < 394730007    20211207    1    2011000195101    900000000000074008
  • > 394730007    20230607    1    2011000195101    900000000000074008

Yet more appear to be added in validly as brand new concepts for this authoring cycle (eg)

  • 62271000195100    20230607    1    2011000195101    900000000000074008
  • 62281000195103    20230607    1    2011000195101    900000000000074008
  • 62291000195101    20230607    1    2011000195101    900000000000074008
  • 72281000195108    20230607    1    2011000195101    900000000000074008
  • 72291000195105    20230607    1    2011000195101    900000000000074008
  • 82291000195104    20230607    1    2011000195101    900000000000074008

However, that leaves 1 active record that has just been removed for no apparent reason:

  • 11000241103    20211207    1    11000241103    900000000000074008

DID RORY PERHAPS INTENTIONALLY MISS OUT ALL CONTENT FOR THIS MODULE (WHEN IMPORTING THE CONTENT INTO THE FRESH NEW CODESYSTEM), BECAUSE THIS IS THE COMMON FRENCH MODULE (11000241103 | SNOMED CT Common French translation module (core metadata concept) |) AND THEREFORE WE'RE NOW GOING FOR AN APPROACH THAT JUST IMPORTS THE COMMON FRENCH CONTENT STRAIGHT INTO THE SWISS MODULE INSTEAD AND THEREFORE THERE IS NO LONGER A DEPENDENCY THERE ON THE COMMON FRENCH MODULE, AND THUS NO REASON TO INCLUDE IT IN THE SWISS PACKAGE????


IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

Yes, Rory confirmed this is correct during our call at 17:00 on  


Description (de-ch) files29593

28030 new records added/updated for 20230607 

1563 records (from all different effectiveTimes) removed

So my manual validation shows:

  1. 1563 records removed from original file   (all in SWISS module (2011000195101))
  2. 28,041 records added back in                       (all in SWISS module (2011000195101))
    1. 99 inactive records (ALL OF THEM) from original Description file had new DescriptionID's assigned to them, ***BUT WERE THEN ADDED BACK IN AS BORN-INACTIVE RECORDS***
    2. 1464 active records were removed from the original Description file
      1. These were all in the Swiss Module (2011000195101), so not sure why we removed these, other than to just wipe the slate clean and start again??
      2. GOOD NEWS is that I spot checked several dozen and they're all JUST DescriptionID REFRESHES, as they all exist in the new content being added back in, so all good as we'd expect the ID's to be refreshed due to this being a "Brand new release"
    3. 28,041 records added in!!! (all with 20230607 effectiveTime, all in SWISS module (2011000195101) 
      1. All in the Swiss Module (2011000195101) as expected
      2. 102 of these records were born-inactive "0" status records
      3. From dozens of spot checks it looks like the 1563 original records were all replaced (just had UUID's refreshed), so we're looking at 26,464 NEW records from the Common German import.
      4. PLUS figures match exactly to the "Desc by Lang" tab of the SCS report above:
        1. 102 "New Inactive"
        2. 27,928 "New" (1464 original active records + 26,464 brand new records) 

 due to close match with TOTAL DESCRIPTION Counts in Summary Component Stats report for this cycle (see link above)

IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

Yes, Rory confirmed this is correct during our call at 17:00 on  

Description (en) files68

40 new records added/updated for 20230607 - 

28 records (from all different effectiveTimes) removed

So my manual validation shows:

  1. 28 records removed from original file   (all in SWISS module (2011000195101))
  2. 40 records added back in                          (all in SWISS module (2011000195101))
    1. 0 inactive records (ALL OF THEM) from original Description file had new DescriptionID's assigned to them
    2. 28 active records were removed from the original Description file
      1. These were all in the Swiss Module (2011000195101), so not sure why we removed these, other than to just wipe the slate clean and start again??
      2. GOOD NEWS is that I spot checked several dozen and they're all JUST DescriptionID REFRESHES, as they all exist in the new content being added back in, so all good as we'd expect the ID's to be refreshed due to this being a "Brand new release"
    3. 40 records added in!!! (all with 20230607 effectiveTime, all in SWISS module (2011000195101) 
      1. All in the Swiss Module (2011000195101) as expected
      2. 0 of these records were born-inactive "0" status records
      3. From dozens of spot checks it looks like the 28 original records were all replaced (just had UUID's refreshed), so we're looking at 12 NEW records from the EN lang.
      4. PLUS figures match exactly to the "Desc by Lang" tab of the SCS report above:
        1. 0 "New Inactive"
        2. 40 "New" (28 original active records + 12 brand new records) 

 due to close match with TOTAL DESCRIPTION Counts in Summary Component Stats report for this cycle (see link above)

IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

Yes, Rory confirmed this is correct during our call at 17:00 on  

Description (fr) files185,262

7479 new records added/updated for 20230607 - 

177,783 records (from all different effectiveTimes) removed

So my manual validation shows:

  1. 177,783 records removed from original file   (all in Common French Module (11000241103))
  2. 7478 (INACTIVE) records added back in        (all in Common French Module (11000241103))
    1. 10,588 inactive records (ALL OF THEM) from original Description file WERE COMPLETELY REMOVED, EXCEPT FOR 7478 OF THEM WHICH WERE ADDED BACK IN WITH NEW descriptionID's???? 
      1. ******** WHY ONLY SOME OF THEM???  WHY ANY OF THEM????!!!  ********
      2. ?????????? as expected ????????????
    2. 167,184 active records were removed from the original Description file
      1. These were all in Common French Module (11000241103), and therefore were completely removed from the Package?????????
    3. 7478 records added in!!! (all with 20230607 effectiveTime, all in SWISS module (2011000195101) 
      1. All in the Common French Module (11000241103) ???????????
      2. 7478 of these records were born-inactive "0" status records, ****** ALL in the Common French Module (11000241103) ******
      3. DOES NOT match figures in the "Desc by Lang" tab of the SCS report above, as the Report seems to have missed out -FR !!!:

 as planned in INFRA-8967 in order to correctly split out the Swiss French records into their own "fr-ch" Description file, leaving ONLY the Common French records in this "fr" Description file.

...as expected due to close match with TOTAL DESCRIPTION Counts in Summary Component Stats report for this cycle (see link above)

IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

NO - RORY CONFIRMED (during our call at 17:00 on ) THAT THE 7478 RECORDS STILL LEFT IN THE CF module (11000241103) WERE JUST A MISTAKE WHEN HE WAS DOING THE ORIGINAL UPLOADS

Description (fr-ch) files158774

158,602 new records added/updated for 20230607 - 

147 records (from all different effectiveTimes) removed

So my manual validation shows:

  1. 147 records removed from original file   (all in SWISS module (2011000195101))
  2. 158,623 records added back in                 (all in SWISS module (2011000195101))
    1. 21 inactive records (ALL OF THEM) from original Description file had new DescriptionID's assigned to them, ***BUT WERE THEN ADDED BACK IN AS BORN-INACTIVE RECORDS***
      1. ?????????? as expected ????????????

      2. YES - RORY CONFIRMED (during our call at 17:00 on ) THAT these born inactive records are expected

    2. 125 active records were removed from the original Description file
      1. These were all in the Swiss Module (2011000195101), so not sure why we removed these, other than to just wipe the slate clean and start again??
      2. GOOD NEWS is that I spot checked several dozen and they're all JUST DescriptionID REFRESHES, as they all exist in the new content being added back in, so all good as we'd expect the ID's to be refreshed due to this being a "Brand new release"
    3. 158,623 records added in!!! (all with 20230607 effectiveTime, all in SWISS module (2011000195101) 
      1. All in the Swiss Module (2011000195101) as expected
      2. 21 of these records were born-inactive "0" status records
      3. From dozens of spot checks it looks like the 125 original records were all replaced (just had UUID's refreshed), so we're looking at 158,477 NEW records from the Common French import.
      4. PLUS figures match exactly to the "Desc by Lang" tab of the SCS report above:
        1. 21 "New Inactive"
        2. 158,602 "New" (125 original active records + 158,477 brand new records) 

...as expected due to close match with TOTAL DESCRIPTION Counts in Summary Component Stats report for this cycle (see link above)

IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

NO - RORY CONFIRMED (during our call at 17:00 on ) THAT THE 7478 RECORDS STILL LEFT IN THE CF module (11000241103) in the description-fr file WERE JUST A MISTAKE WHEN HE WAS DOING THE ORIGINAL UPLOADS.  They therefore now need to be added back into this description-fr-ch file in the Swiss module (2011000195101).

Description (it-ch) files2637

1326 new records added/updated for 20230607 - 

1311 records (from all different effectiveTimes) removed

So my manual validation shows:

  1. 1311 records removed from original file   (all in SWISS module (2011000195101))
  2. 1326 records added back in                          (all in SWISS module (2011000195101))
    1. 32 inactive records (ALL OF THEM) from original Description file had new DescriptionID's assigned to them
    2. 1279 active records were removed from the original Description file
      1. These were all in the Swiss Module (2011000195101), so not sure why we removed these, other than to just wipe the slate clean and start again??
      2. GOOD NEWS is that I spot checked several dozen and they're all JUST DescriptionID REFRESHES, as they all exist in the new content being added back in, so all good as we'd expect the ID's to be refreshed due to this being a "Brand new release"
    3. 40 records added in!!! (all with 20230607 effectiveTime, all in SWISS module (2011000195101) 
      1. All in the Swiss Module (2011000195101) as expected
      2. 34 of these records were born-inactive "0" status records
      3. From dozens of spot checks it looks like the 1279 original active records were all replaced (just had UUID's refreshed), so we're looking at 13 NEW records from the IT lang.
      4. PLUS figures match exactly to the "Desc by Lang" tab of the SCS report above:
        1. 34 "New Inactive"
        2. 1292 "New" (1279 original active records + 13 brand new records) 

 due to close match with TOTAL DESCRIPTION Counts in Summary Component Stats report for this cycle (see link above)

IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

Yes, Rory confirmed this is correct during our call at 17:00 on  

Relationship files64

37 new records added/updated for 20230607 - 

27 records (from all different effectiveTimes) removed

...NOT as expected due to

...

all x prefixes added in, plus PreProduction naming convention applied, as expected. 

ALL Delta files removed as well, as expected.

PLUS 2x new file types added in as planned, meaning 4x new files in total added:

  • > xsct2_Description_Full-fr-ch_CH1000195_20220607.txt
  • > xsct2_Description_Snapshot-fr-ch_CH1000195_20220607.txt
  • > xsct2_TextDefinition_Full-fr_CH1000195_20220607.txt
  • > xsct2_TextDefinition_Snapshot-fr_CH1000195_20220607.txt

...

1 new record added for new "Romansh [International Organization for Standardization 639-1 code rm] language reference set (foundation metadata concept)" concept:

> 22241000195100 20220607 1 2011000195101 900000000000074008

VERIFYING THIS WITH PERO IN MSSP-1518, as although all of the relevant components are here as expected (concepts, descriptions in all languages, etc) - there's NO new actual language refset file?!!

Plus 1 inactivated record:

< 394812008 20211207 1 2011000195101 900000000000074008
> 394812008 20220607 0 2011000195101 900000000000074008

...

Summary Component Stats report for this cycle (see link above)

...

showing:

  • 37

...

  • new records added/updated

...

...as expected due to exact match in Summary Component Stats report for this cycle (see link above)

...

Image RemovedISRS-1254

10840 added/updated??!!!

NOT AS EXPECTED!!! SUMMARY COMPONENT REPORT SHOWS 88 ADDED EXPECTED, WHICH IS MUCH CLOSER TO EXPECTED THAN 10000+!!!!!

10753 records have been added here for the Common French descriptions (11000241103) - question is why?!!

The snapshot of this file had only 20 records in it for the December 2021 release!

However when I run the June 2022 release, it now has over 10000 records!  (there’s only been 200 odd Description changes in this cycle!)

...

To be fair, looking at the Common French Description file (https://prod-release.ihtsdotools.org/api/centers/ch/products/snomed_ct_switzerland_releases[…]s/xsct2_Description_Snapshot-fr_CH1000195_20220607.txt), it looks like there were about 10254 records inactivated in the December release - however these were Born Inactive (in the first time release) and so I wouldn’t expect the system to create new attributeValue records for these?  But perhaps it has to if it can’t find existing attributeValue records for the old inactive records??

CHECKING WITH QUYEN...Quyen thinks it's a snowstorm bug, so I've raised ticket:

Jira
serverIHTSDO JIRA
serverIdb202d822-d767-33be-b234-fec5accd5d8c
keyISRS-1254

Michael confirmed them to be all valid in ISRS-1254

  • for 20230607, 

  • 0 records inactivated

Some of the records removed appear to be absolutely valid, as they are previously inactive records (and therefore there's no need to keep them as "born inactive" records) (eg)

  • 2026638020    20220607    0    2011000195101    394812008    394733009    0    116680003    900000000000011006    900000000000451002
  • 11000241126    20220607    0    2011000195101    11000241103    900000000000443000    0    116680003    900000000000011006    900000000000451002

Many more appear to be valid as they're simply "removing" these records to be replaced with the same record just with new effectiveTimes (eg)

  • < 33588028    20211207    1    2011000195101    224877003    308912000    0    116680003    900000000000011006    900000000000451002
  • > 33588028    20230607    1    2011000195101    224877003    308912000    0    116680003    900000000000011006    900000000000451002

Yet more appear to be added in validly as brand new concepts for this authoring cycle (eg)

  • > 123431000195124    20230607    1    2011000195101    72281000195108    365996006    0    116680003    900000000000011006    900000000000451002
  • > 123441000195127    20230607    1    2011000195101    72281000195108    225598002    1    363714003    900000000000011006    900000000000451002
  • > 123481000195123    20230607    1    2011000195101    82291000195104    417319006    0    116680003    900000000000011006    900000000000451002
  • > 123501000195126    20230607    1    2011000195101    62271000195100    364830008    1    363702006    900000000000011006    900000000000451002

The good news here is that the only record related to the now unnecessary Common French module (11000241103) was already inactive, and therefore gets removed anyway:

  • < 11000241126    20220607    0    2011000195101    11000241103    900000000000443000    0    116680003    900000000000011006    900000000000451002

IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

Yes, Rory confirmed this is correct during our call at 17:00 on  

Relationship Concrete Values files0

0 records added/updated

...as expected due to exact match in Summary Component Stats report for this cycle (see link above)

OWL Expression files46

25 new records added/updated for 20230607 - 

21 records (from all different effectiveTimes) removed

...as expected ????????????????

Some of the records removed appear to be absolutely valid, as they are previously inactive records (and therefore there's no need to keep them as "born inactive" records) (eg)

  • 1ed01c6c-26ea-4b1f-a59e-c79fd882fc51    20220607    0    2011000195101    733073007    394812008    SubClassOf(:394812008 :394733009)

Many more appear to be valid as they're simply "removing" these records to be replaced with the same record just with new effectiveTimes (eg)

  • < 07806988-6dec-40d9-b4b9-53b4cd6458bd    20221207    1    2011000195101    733073007    52281000195106    SubClassOf(:52281000195106 :229824005)
  • > 07806988-6dec-40d9-b4b9-53b4cd6458bd    20230607    1    2011000195101    733073007    52281000195106    SubClassOf(:52281000195106 :229824005)

The good news here is that one of the records related to the now unnecessary Common French module (11000241103) was already inactive, and therefore gets removed anyway:

  • < 11000241126    20220607    0    2011000195101    11000241103    900000000000443000    0    116680003    900000000000011006    900000000000451002

The other one relating to this module also gets removed, as expected:

  • < b7c3c7f7-1967-0917-6c22-db19ddd83319    20211207    1    11000241103    733073007    11000241103    SubClassOf(:11000241103 :900000000000443000)

Yet more appear to be added in validly as brand new concepts for this authoring cycle (eg)

  • > 28b8c71c-7f57-46d1-af19-9344a5f47bef    20230607    1    2011000195101    733073007    72281000195108    SubClassOf(:72281000195108 :365996006)
  • > 3a3153be-35f2-49c4-ab6e-acebbc172227    20230607    1    2011000195101    733073007    62291000195101    SubClassOf(:62291000195101 :229824005)
  • > 699639a9-0d0f-4b43-8cb0-bf78e56e8c26    20230607    1    2011000195101    733073007    62281000195103    SubClassOf(:62281000195103 :229824005)
  • > 781e5e06-349f-402b-93e4-c9e77b193836    20230607    1    2011000195101    733073007    82291000195104    SubClassOf(:82291000195104 :417319006)
  • > eb98a3ab-c77b-411c-ad04-2965d66da434    20230607    1    2011000195101    733073007    72291000195105    SubClassOf(:72291000195105 :365508006)
  • > fac07ccd-6653-43d7-bdd1-d51d9d586a58    20230607    1    2011000195101    733073007    62271000195100    SubClassOf(:62271000195100 ObjectIntersectionOf(:229824005 :371735007))

IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

Yes, Rory confirmed this is correct during our call at 17:00 on  

TextDefinition (de-ch) files0

0 new records added/updated for 20230607 - 

0 records (from all different effectiveTimes) removed

...as expected as this file was empty in the previous releases.

TextDefinition (en) files0

0 new records added/updated for 20230607 - 

0 records (from all different effectiveTimes) removed

...as expected as this file was empty in the previous releases.

TextDefinition (fr) files1

0 new records added/updated for 20230607 - 

1 records (from all different effectiveTimes) removed

1 record related to the now unnecessary Common French module (11000241103) was removed as expected:

1020231000241112    20220331    1    11000241103    77386006    fr    900000000000550004    Patiente enceinte actuellement    900000000000017005

IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

Yes, Rory confirmed this is correct during our call at 17:00 on  

TextDefinition (fr-ch) files0

0 new records added/updated for 20230607 - 

0 records (from all different effectiveTimes) removed

...as expected as this file was empty in the previous releases.

TextDefinition (it-ch) files0

0 new records added/updated for 20230607 - 

0 records (from all different effectiveTimes) removed

...as expected as this file was empty in the previous releases

...

74 added/updated records + 

41 inactivated records

...as expected due to close match with TOTAL DESCRIPTION Counts in Summary Component Stats report for this cycle (see link above)

...

2 new records added/updated

...as expected due to close match with TOTAL DESCRIPTION Counts in Summary Component Stats report for this cycle (see link above)

...

110 Swiss French records removed, as planned in INFRA-8967 in order to correctly split out the Swiss French records into their own "fr-ch" Description file, leaving ONLY the Common French records in this "fr" Description file.

...as expected due to close match with TOTAL DESCRIPTION Counts in Summary Component Stats report for this cycle (see link above)

...

Original 110 Swiss French records added, as planned in INFRA-8967 in order to correctly split out the Swiss French records into their own "fr-ch" Description file, leaving ONLY the Common French records in this "fr" Description file.

+ 21 records inactivated in the new June 2022 content

+ 30 new records added in the new June 2022 content

...as expected due to close match with TOTAL DESCRIPTION Counts in Summary Component Stats report for this cycle (see link above)

...

61 added/updated records + 

22 inactivated records

...as expected due to close match with TOTAL DESCRIPTION Counts in Summary Component Stats report for this cycle (see link above)

...

71 records added/updated

+ 41 records inactivated

...as expected due to exact match in Summary Component Stats report for this cycle (see link above)

...

68 records added/updated

+ 22 records inactivated

...as expected due to exact match in Summary Component Stats report for this cycle (see link above)

...

67 records added/updated

...NOT as expected due to Summary Component Stats report for this cycle (see link above) expecting 40????

Double checked and this looks like a problem with the Summary Component Report - possibly just because we're comparing to a DIFFERENT previous published package though, because we've generated the new one?!!

...

0 records added/updated, as expected as this is Common French Language file??????

(but then shouldn't we be moving some into fr-ch, or were these already in December - no they weren't there, they requested no preferences for Common French, just Swiss French)

SUMMARY REPORT EXPECTING 11 - this looks like a problem with the Summary Component Report - possibly just because we're comparing to a DIFFERENT previous published package though, because we've generated the new one?!! 

...

67 records added/updated

+ 89 records inactivated

...as expected due to exact match in Summary Component Stats report for this cycle (see link above)

...

1 new records added:

> 43341000195128 20220607 1 2011000195101 22241000195100 900000000000506000 0 116680003 900000000000011006 900000000000451002

plus 2 records inactivated, related to newly re-activated Concepts:

< 11000241126 20211207 1 11000241103 11000241103 900000000000443000 0 116680003 900000000000011006 900000000000451002
> 11000241126 20220607 0 2011000195101 11000241103 900000000000443000 0 116680003 900000000000011006 900000000000451002

< 2026638020 20211207 1 2011000195101 394812008 394733009 0 116680003 900000000000011006 900000000000451002
> 2026638020 20220607 0 2011000195101 394812008 394733009 0 116680003 900000000000011006 900000000000451002

NOT AS EXPECTED - THE SUMMARY COMPONENT REPORT (linked) SHOWS 2 new and 1 inactivated expected????)  Maybe because it's comparing to a DIFFERENT previous published package though, because we've generated the new one?!!  Yes, manual checks show 2 inactivations + 1 addition since previous publication...

...

0 records added/updated

...as expected due to exact match in Summary Component Stats report for this cycle (see link above)

...

1 new records added:

> 5a9f6ee1-5e4a-4e6d-b740-e123946cfef6 20220607 1 2011000195101 733073007 22241000195100 SubClassOf(:22241000195100 :900000000000506000)

plus 1 records inactivated, related to newly re-activated Concepts:

< 1ed01c6c-26ea-4b1f-a59e-c79fd882fc51 20211207 1 2011000195101 733073007 394812008 SubClassOf(:394812008 :394733009)
> 1ed01c6c-26ea-4b1f-a59e-c79fd882fc51 20220607 0 2011000195101 733073007 394812008 SubClassOf(:394812008 :394733009)

...

.