SNOMED International Release Management

Page tree

Versions Compared

Old Version 6

changes.mady.by.user Andrew Atkinson

Saved on

compared with

New Version Current

changes.mady.by.user Andrew Atkinson

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

June 2023 PreProduction (Version 5 - Fixes from version 4) traceability vs June 2023

...

Production (Version

...

1) traceability

SUMMARY COMPONENT REPORT:  https://docs.google.com/spreadsheets/d/1Q18jq4eHU4-H8hou3U3Pb-4l1jGgqRNSIk-1KvTfrIo/edit#gid=1

MSSP TICKET:  https://jira.ihtsdotools.org/browse/MSSP-2072

Differences found in package Comparison

Number of RF2 records impactedRelated JIRA ticket(s)Rationale

...

Readme filen/a

All x prefixes removed, and Production naming conventions applied, as expected


June 2023 PreProduction (Version 4 - COMPLETE RE-START OF SWISS RELEASE, AS IF THIS IS A "FIRST TIME" RELEASE

...

) traceability

 vs June 2023 PreProduction (Version 5 - Fixes from version 4) traceability

SUMMARY COMPONENT REPORT:  https://docs.google.com/spreadsheets/d/1Q18jq4eHU4-H8hou3U3Pb-4l1jGgqRNSIk-1KvTfrIo/edit#gid=1

MSSP TICKET:  https://jira.ihtsdotools.org/browse/MSSP-2072

Differences found in package Comparison

Number of RF2 records impactedRelated JIRA ticket(s)Rationale
RefsetDescriptor files10

8 records had their UUID's reverted to December 2022 (plus 2 records that were missing from the PreProd version 4 package) + new effectiveTime of 20230607 (but NO CHANGES TO ACTUAL RECORDS) as expected, due to NEW "FIRST TIME" RELEASE:

  • < 548dc527-855a-476a-9e19-183666f89c8d    20230607

...

  •    1    2011000195101    900000000000456007    2031000195108    900000000000511003    900000000000461009    1
    <

...

  • c1f1334c-

...

  • 9bb0-

...

  • 4149-

...

  • a2ea-

...

  • 00914853e3cf  

...

  •  20230607    1    2011000195101    900000000000456007  

...

  •  22241000195100  

...

  •  900000000000510002    900000000000462002    0
    < c6263863-9152-4e93-b229-d8b0243d8e6b    20230607    1    2011000195101    900000000000456007    2021000195106    900000000000511003    900000000000461009    1

...

  • < d9196a6b-29a1-4dbd-bb6b-9346429d9dfc    20230607    1    2011000195101    900000000000456007    2041000195100    900000000000511003    900000000000461009    1

...

  • < e654efc1-6113-4eff-acb9-a08ade12dc7e    20230607    1    2011000195101    900000000000456007    2041000195100    900000000000510002    900000000000462002    0

...

  • < eb71ee49-88b2-4713-950a-4506e714b456    20230607    1    2011000195101    900000000000456007    2031000195108    900000000000510002    900000000000462002    0

...

  • < eb9f152b-79f1-4d3f-8b8a-6381cb076d45    20230607    1    2011000195101    900000000000456007    22241000195100    900000000000511003    900000000000461009    1

...

  • < fdf18dfc-59ac-4ef6-a844-9b883e9dc531    20230607    1    2011000195101    900000000000456007    2021000195106    900000000000510002    900000000000462002    0

2 records however, DID NOT GET REPLACED IN NEW JUNE 2023 RELEASE as expected???

  • > 174062a5-02eb-4179-899c-741faa68d9a6    20230607    1    2011000195101    900000000000456007  

...

  •  2031000195108    900000000000510002    900000000000462002    0

...

  • > 1ca3e72a-

...

  • 71ee-

...

  • 436d-

...

  • 9c06-

...

  • e47e0db53185  

...

  •  20230607    1    2011000195101    900000000000456007  

...

  •  22241000195100  

...

  •  900000000000510002  

...

  •  900000000000462002  

...

5 records removed completey, replaced with nothing!!

SO ALL WRONG!! NEED TO REVERT BACK TO EXTERNALLYMAINTAINED INPUT FOR THIS FILE TO SEE IF THIS FIXES IT!!

  •  0
    > 1dda242a-587b-4b38-8eff-f86b509afa22    20230607    1    2011000195101    900000000000456007    22241000195100    900000000000511003    900000000000461009    1
    > 2086b060-a11f-4409-867c-56c308f356ea    20230607    1    2011000195101    900000000000456007    2041000195100    900000000000511003    900000000000461009    1
    > 31a3a105-910b-4c7f-842d-1b0e7b9b89de    20230607    1    2011000195101    900000000000456007    2021000195106    900000000000510002    900000000000462002    0
    > 4ae80dc6-2654-4188-85c9-4e5003c96c6f    20230607    1    2011000195101    900000000000456007    2041000195100    900000000000510002    900000000000462002    0
    > 58b37409-0e3b-49ec-9cc4-f2ea835a5789    20230607    1    2011000195101    900000000000456007    722131000    900000000000510002    900000000000462002    0
    > 8afd2ce3-9943-4359-bf0a-18f812ae1bbd    20230607    1    2011000195101    900000000000456007    2031000195108    900000000000511003    900000000000461009    1
    > b9ca50ba-2a5e-4cda-806b-aa313a84dcf9    20230607    1    2011000195101    900000000000456007    722131000    900000000000511003    900000000000461009    1
    > ef7acdd0-afcd-416b-8968-358b8d5dfa88    20230607    1    2011000195101    900000000000456007    2021000195106    900000000000511003    900000000000461009    1

So when compared to the old December 2022 release, the new Version 5 refsetDescriptor records are identical except for the new effectiveTime of 20230607 for each record, as expected.

ModuleDependency files2

2 records added back in as expected, as remaining 3 from December 2022 were all related to the old dependency on the CF module, which no longer exists!

  • > 36ddb438-742a-4c97-9dca-b95489b6a08a    20230607    1    2011000195101    900000000000534007    900000000000207008    20230607    20230430
  • > a4fbf40a-73ae-4e51-80da-cddc448d3a32    20230607    1    2011000195101    900000000000534007    900000000000012004    20230607    20230430

Therefore, when compared to the old December 2022 release, the new Version 5 moduleDependency records are identical (except for the new effectiveTime of 20230607 for each record) for the 2 records that are still applicable in the new world of hosting all CF + CG records in the CH module instead of in the CF + CG modules.  However, we've now removed 3 of the old moduleDependency records, as we no longer need to call out the dependencies on those CF + CG modules:

  • b3c09a5c-496c-48d7-808a-166f2b6a3c97    20221207    1    11000241103    900000000000534007    900000000000207008    20221031    20221031

  • dc968531-82da-4aeb-bf59-3ae96cf71dd7    20221207    1    2011000195101    900000000000534007    11000241103    20221207    20221031

  • 7321d59a-986d-4496-9cf8-874022470ad5    20221207    1    11000241103    900000000000534007    900000000000012004    20221031    20221031

This is all now exactly as expected.

Description-fr FULL + SNAPSHOT files7478

7478 inactive CF module (11000241103) records have been completely removed from the package.

This is exactly as expected, due to the fixes that were applied by Rory after the remaining 7478 CF module records were identified in the Version 4 testing below...

Description-fr-ch FULL + SNAPSHOT files7478

7478 inactive records (from the -fr file, that were removed because they were still in the CF module (11000241103)) have now been added back into the package in the RIGHT MODULE (the Swiss module 2011000195101)

This is exactly as expected, due to the fixes that were applied by Rory after the remaining 7478 CF module records were identified in the Version 4 testing below...


December 2022 Production (PUBLISHED) vs June 2023 PreProduction (Version 4 - COMPLETE RE-START OF SWISS RELEASE, AS IF THIS IS A "FIRST TIME" RELEASE) traceability

SUMMARY COMPONENT REPORT:  https://docs.google.com/spreadsheets/d/1Q18jq4eHU4-H8hou3U3Pb-4l1jGgqRNSIk-1KvTfrIo/edit#gid=1

MSSP TICKET:  https://jira.ihtsdotools.org/browse/MSSP-2072

Differences found in package Comparison

Number of RF2 records impactedRelated JIRA ticket(s)Rationale
RefsetDescriptor files10

8 records had their UUID's refreshed + new effectiveTime of 20230607 (but NO CHANGES TO ACTUAL RECORDS) as expected, due to NEW "FIRST TIME" RELEASE:

  • Dec 22
    • < 174062a5-02eb-4179-899c-741faa68d9a6    20211207    1    2011000195101    900000000000456007    2031000195108    900000000000510002    900000000000462002    0
    • < 1ca3e72a-71ee-436d-9c06-e47e0db53185    20220607    1    2011000195101    900000000000456007    22241000195100    900000000000510002    900000000000462002    0
    • < 1dda242a-587b-4b38-8eff-f86b509afa22    20220607    1    2011000195101    900000000000456007    22241000195100    900000000000511003    900000000000461009    1
    • < 2086b060-a11f-4409-867c-56c308f356ea    20211207    1    2011000195101    900000000000456007    2041000195100    900000000000511003    900000000000461009    1
    • < 31a3a105-910b-4c7f-842d-1b0e7b9b89de    20211207    1    2011000195101    900000000000456007    2021000195106    900000000000510002    900000000000462002    0
    • < 4ae80dc6-2654-4188-85c9-4e5003c96c6f    20211207    1    2011000195101    900000000000456007    2041000195100    900000000000510002    900000000000462002   0
    • < 8afd2ce3-9943-4359-bf0a-18f812ae1bbd    20211207    1    2011000195101    900000000000456007    2031000195108    900000000000511003    900000000000461009    1
    • < ef7acdd0-afcd-416b-8968-358b8d5dfa88    20211207    1    2011000195101    900000000000456007    2021000195106    900000000000511003    900000000000461009    1
  • June 23
    • > 548dc527-855a-476a-9e19-183666f89c8d    20230607    1    2011000195101    900000000000456007    2031000195108    900000000000511003    900000000000461009    1
    • > c1f1334c-9bb0-4149-a2ea-00914853e3cf    20230607    1    2011000195101    900000000000456007    22241000195100    900000000000510002    900000000000462002    0
    • > c6263863-9152-4e93-b229-d8b0243d8e6b    20230607    1    2011000195101    900000000000456007    2021000195106    900000000000511003    900000000000461009    1
    • > d9196a6b-29a1-4dbd-bb6b-9346429d9dfc    20230607    1    2011000195101    900000000000456007    2041000195100    900000000000511003    900000000000461009    1
    • > e654efc1-6113-4eff-acb9-a08ade12dc7e    20230607    1    2011000195101    900000000000456007    2041000195100    900000000000510002    900000000000462002    0
    • > eb71ee49-88b2-4713-950a-4506e714b456    20230607    1    2011000195101    900000000000456007    2031000195108    900000000000510002    900000000000462002    0
    • > eb9f152b-79f1-4d3f-8b8a-6381cb076d45    20230607    1    2011000195101    900000000000456007    22241000195100    900000000000511003    900000000000461009    1
    • > fdf18dfc-59ac-4ef6-a844-9b883e9dc531    20230607    1    2011000195101    900000000000456007    2021000195106    900000000000510002    900000000000462002    0

2 records however, DID NOT GET REPLACED IN NEW JUNE 2023 RELEASE as expected???

  • < 58b37409-0e3b-49ec-9cc4-f2ea835a5789    20211207    1    2011000195101    900000000000456007    722131000    900000000000510002    900000000000462002    0
  • < b9ca50ba-2a5e-4cda-806b-aa313a84dcf9    20211207    1    2011000195101    900000000000456007    722131000    900000000000511003    900000000000461009    1


ModuleDependency files5

5 records removed completey, replaced with nothing!!

SO ALL WRONG!! NEED TO REVERT BACK TO EXTERNALLYMAINTAINED INPUT FOR THIS FILE TO SEE IF THIS FIXES IT!!

Readme file

all x prefixes added in, plus PreProduction naming convention applied, as expected. 

Association files12

2 new records added/updated for 20230607, 

10 records (from all different effectiveTimes) removed

...NOT as expected due to Summary Component Stats report for this cycle (see link above) showing:

  • 2 new records added/updated for 20230607, 

  • 0 records inactivated

IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

ALL HISTORY GONE FROM ASSOCIATION FILES - I GUESS THIS IS WHAT WE WERE AIMING FOR WITH A BRAND NEW RELEASE, BUT JUST NEED TO DOUBLE CHECK WITH RORY...

Yes, Rory confirmed this is correct during our call at 17:00 on  

AttributeValue files11613

6860 new records added/updated for 20230607, 

11613 records (from all different effectiveTimes) removed

...NOT as expected due to Summary Component Stats report for this cycle (see link above) showing:

  • 6860 new records added/updated for 20230607, 

  • 0 records inactivated

IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

ALL HISTORY GONE FROM ATTRIBUTEVALUE FILES - I GUESS THIS IS WHAT WE WERE AIMING FOR WITH A BRAND NEW RELEASE, BUT JUST NEED TO DOUBLE CHECK WITH RORY...

Yes, Rory confirmed this is correct during our call at 17:00 on  

Language Refset (de-ch) files29491

27928 new records added/updated for 20230607 - ***** FOR REFSETID: 2041000195100 *****

1563 records (from all different effectiveTimes) removed

..NOT as expected due to Summary Component Stats report for this cycle (see link above) showing:

  • 27928 new records added/updated for 20230607, 

  • 0 records inactivated

DOES THIS MEAN THE SCS REPORT WILL BE INVALID, AS WILL SHOW NOW INACTIVATIONS BECAUSE IT'S A "BRAND NEW RELEASE" WITH NO "PREVIOUSPUBLISHEDPACKAGE" TO COMPARE THE CONTENT TO????

So my manual validation shows:

  1. 99 inactive records (ALL OF THEM) were removed from the original Language refset file, as expected
  2. 1465 active records were removed from the original Language refset file, all in refset: 2041000195100 (Swiss German LRS)
    1. These were all in the Swiss Module (2011000195101), so not sure why we removed these, other than to just wipe the slate clean and start again??
    2. GOOD NEWS is that I spot checked several dozen and they're all JUST UUID REFRESHES, as they all exist in the new content being added back in, so all good as we'd expect the UUID's to be refreshed due to this being a "Brand new release"
  3. 27,928 records added back in!!!
    1. All in the Swiss Module (2011000195101) as expected
    2. So assuming that the 1465 original records were all replaced (just had UUID's refreshed), we're looking at 26,463 NEW records from the Common German import.
    3. PLUS 27,928 matches exactly to the SCS report above

IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

Yes, Rory confirmed this is correct during our call at 17:00 on  

Language Refset (it-ch) files2603

1292 new records added/updated for 20230607 - ***** FOR REFSETID: 2031000195108 *****

1311 records (from all different effectiveTimes) removed

So my manual validation shows:

  1. 32 inactive records (ALL OF THEM) were removed from the original Language refset file, as expected
  2. 1279 active records were removed from the original Language refset file, all in refset: 2031000195108 (Swiss Italian LRS)
    1. This accounts for the removal of ALL it-ch records from previous package, as expected as inactives removed forever, and actives removed just to refresh UUID's
  3. 1292 records added back in!! (all with 20230607 effectiveTime, all in SWISS module (2011000195101) + all in refset: 2031000195108 (Swiss Italian LRS)
    1. From dozens of spot checks this looks like 1279 original active records (just has UUID's refreshed) + 13 NEW records for this cycle
    2. PLUS 1292 matches exactly to the SCS report above

...as expected ????????????????

Yes, Rory confirmed this is correct during our call at 17:00 on  

Language Refset (en) files182

96 new records added/updated for 20230607 - ***** FOR REFSETID: 900000000000508004 + 900000000000509007 *****

86 records (from all different effectiveTimes) removed

So my manual validation shows:

  1. 0 inactive records (ALL OF THEM) were removed from the original Language refset file, as expected
  2. 86 active records were removed from the original Language refset file, all in refset: 900000000000509007 (en-US LRS) + 900000000000508004 (en-GB LRS)
    1. This accounts for the removal of ALL it-ch records from previous package, as expected as inactives removed forever, and actives removed just to refresh UUID's
  3. 98 records added back in!! (all with 20230607 effectiveTime, all in SWISS module (2011000195101) + all in refset: 900000000000509007 (en-US LRS) + 900000000000508004 (en-GB LRS)
    1. From dozens of spot checks this looks like 86 original active records (just has UUID's refreshed) + 12 NEW records for this cycle
    2. PLUS 98 matches exactly to the SCS report above

IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

Yes, Rory confirmed this is correct during our call at 17:00 on  

Language Refset (fr) files0

0 records added/updated, as expected as this is Common French Language file??????

SUMMARY REPORT EXPECTING 0

Language Refset (fr-ch) files260,288

161,454 new records added/updated for 20230607 - ***** FOR REFSETID: 2021000195106 *****

98,834 records (from all different effectiveTimes) removed

So my manual validation shows:

  1. 6059 inactive records (ALL OF THEM) were removed from the original Language refset file, as expected
  2. 92,775 active records were removed from the original Language refset file, all in refset: 2021000195106 (Swiss French LRS)
    1. These were all in the Swiss Module (2011000195101), so not sure why we removed these, other than to just wipe the slate clean and start again??
      1. ******* BUT I THOUGHT WE WERE DOING SOMETHING DIFFERENT IN THIS CYCLE WITH TEH COMMON FRENCH APPROACH??  MAYBE JUST IN DESCRIPTIONS??????? *******
    2. GOOD NEWS is that I spot checked several dozen and they're all JUST UUID REFRESHES, as they all exist in the new content being added back in, so all good as we'd expect the UUID's to be refreshed due to this being a "Brand new release"
  3. 161,454 records added back in!!! (all with 20230607 effectiveTime, all in SWISS module (2011000195101) + all in refset: 2021000195106 (Swiss French LRS)
    1. All in the Swiss Module (2011000195101) as expected
    2. From dozens of spot checks it looks like the 92,775 original records were all replaced (just had UUID's refreshed), so we're looking at 68,679 NEW records from the Common French import.
    3. PLUS 161,454 matches exactly to the SCS report above

IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

Yes, Rory confirmed this is correct during our call at 17:00 on  

Concept files46

25 new records added/updated for 20230607, 

21 records (from all different effectiveTimes) removed

...NOT as expected due to Summary Component Stats report for this cycle (see link above) showing:

  • 25 new records added/updated for 20230607, 

  • 0 records inactivated

DOES THIS MEAN THE SCS REPORT WILL BE INVALID, AS WILL SHOW NOW INACTIVATIONS BECAUSE IT'S A "BRAND NEW RELEASE" WITH NO "PREVIOUSPUBLISHEDPACKAGE" TO COMPARE THE CONTENT TO????

Some of the records removed appear to be absolutely valid, as they are previously inactive records (and therefore there's no need to keep them as "born inactive" records) (eg)

  • 394812008    20220607    0    2011000195101    900000000000074008

Many more appear to be valid as they're simply "removing" these records to be replaced with the same record just with new effectiveTimes (eg)

  • < 394730007    20211207    1    2011000195101    900000000000074008
  • > 394730007    20230607    1    2011000195101    900000000000074008

Yet more appear to be added in validly as brand new concepts for this authoring cycle (eg)

  • 62271000195100    20230607    1    2011000195101    900000000000074008
  • 62281000195103    20230607    1    2011000195101    900000000000074008
  • 62291000195101    20230607    1    2011000195101    900000000000074008
  • 72281000195108    20230607    1    2011000195101    900000000000074008
  • 72291000195105    20230607    1    2011000195101    900000000000074008
  • 82291000195104    20230607    1    2011000195101    900000000000074008

However, that leaves 1 active record that has just been removed for no apparent reason:

  • 11000241103    20211207    1    11000241103    900000000000074008

DID RORY PERHAPS INTENTIONALLY MISS OUT ALL CONTENT FOR THIS MODULE (WHEN IMPORTING THE CONTENT INTO THE FRESH NEW CODESYSTEM), BECAUSE THIS IS THE COMMON FRENCH MODULE (11000241103 | SNOMED CT Common French translation module (core metadata concept) |) AND THEREFORE WE'RE NOW GOING FOR AN APPROACH THAT JUST IMPORTS THE COMMON FRENCH CONTENT STRAIGHT INTO THE SWISS MODULE INSTEAD AND THEREFORE THERE IS NO LONGER A DEPENDENCY THERE ON THE COMMON FRENCH MODULE, AND THUS NO REASON TO INCLUDE IT IN THE SWISS PACKAGE????


IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

Yes, Rory confirmed this is correct during our call at 17:00 on  


Description (de-ch) files29593

28030 new records added/updated for 20230607 

1563 records (from all different effectiveTimes) removed

So my manual validation shows:

  1. 1563 records removed from original file   (all in SWISS module (2011000195101))
  2. 28,041 records added back in                       (all in SWISS module (2011000195101))
    1. 99 inactive records (ALL OF THEM) from original Description file had new DescriptionID's assigned to them, ***BUT WERE THEN ADDED BACK IN AS BORN-INACTIVE RECORDS***
    2. 1464 active records were removed from the original Description file
      1. These were all in the Swiss Module (2011000195101), so not sure why we removed these, other than to just wipe the slate clean and start again??
      2. GOOD NEWS is that I spot checked several dozen and they're all JUST DescriptionID REFRESHES, as they all exist in the new content being added back in, so all good as we'd expect the ID's to be refreshed due to this being a "Brand new release"
    3. 28,041 records added in!!! (all with 20230607 effectiveTime, all in SWISS module (2011000195101) 
      1. All in the Swiss Module (2011000195101) as expected
      2. 102 of these records were born-inactive "0" status records
      3. From dozens of spot checks it looks like the 1563 original records were all replaced (just had UUID's refreshed), so we're looking at 26,464 NEW records from the Common German import.
      4. PLUS figures match exactly to the "Desc by Lang" tab of the SCS report above:
        1. 102 "New Inactive"
        2. 27,928 "New" (1464 original active records + 26,464 brand new records) 

 due to close match with TOTAL DESCRIPTION Counts in Summary Component Stats report for this cycle (see link above)

IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

Yes, Rory confirmed this is correct during our call at 17:00 on  

Description (en) files68

40 new records added/updated for 20230607 - 

28 records (from all different effectiveTimes) removed

So my manual validation shows:

  1. 28 records removed from original file   (all in SWISS module (2011000195101))
  2. 40 records added back in                          (all in SWISS module (2011000195101))
    1. 0 inactive records (ALL OF THEM) from original Description file had new DescriptionID's assigned to them
    2. 28 active records were removed from the original Description file
      1. These were all in the Swiss Module (2011000195101), so not sure why we removed these, other than to just wipe the slate clean and start again??
      2. GOOD NEWS is that I spot checked several dozen and they're all JUST DescriptionID REFRESHES, as they all exist in the new content being added back in, so all good as we'd expect the ID's to be refreshed due to this being a "Brand new release"
    3. 40 records added in!!! (all with 20230607 effectiveTime, all in SWISS module (2011000195101) 
      1. All in the Swiss Module (2011000195101) as expected
      2. 0 of these records were born-inactive "0" status records
      3. From dozens of spot checks it looks like the 28 original records were all replaced (just had UUID's refreshed), so we're looking at 12 NEW records from the EN lang.
      4. PLUS figures match exactly to the "Desc by Lang" tab of the SCS report above:
        1. 0 "New Inactive"
        2. 40 "New" (28 original active records + 12 brand new records) 

 due to close match with TOTAL DESCRIPTION Counts in Summary Component Stats report for this cycle (see link above)

IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

Yes, Rory confirmed this is correct during our call at 17:00 on  

Description (fr) files185,262

7479 new records added/updated for 20230607 - 

177,783 records (from all different effectiveTimes) removed

So my manual validation shows:

  1. 177,783 records removed from original file   (all in Common French Module (11000241103))
  2. 7478 (INACTIVE) records added back in        (all in Common French Module (11000241103))
    1. 10,588 inactive records (ALL OF THEM) from original Description file WERE COMPLETELY REMOVED, EXCEPT FOR 7478 OF THEM WHICH WERE ADDED BACK IN WITH NEW descriptionID's???? 
      1. ******** WHY ONLY SOME OF THEM???  WHY ANY OF THEM????!!!  ********
      2. ?????????? as expected ????????????
    2. 167,184 active records were removed from the original Description file
      1. These were all in Common French Module (11000241103), and therefore were completely removed from the Package?????????
    3. 7478 records added in!!! (all with 20230607 effectiveTime, all in SWISS module (2011000195101) 
      1. All in the Common French Module (11000241103) ???????????
      2. 7478 of these records were born-inactive "0" status records, ****** ALL in the Common French Module (11000241103) ******
      3. DOES NOT match figures in the "Desc by Lang" tab of the SCS report above, as the Report seems to have missed out -FR !!!:

 as planned in INFRA-8967 in order to correctly split out the Swiss French records into their own "fr-ch" Description file, leaving ONLY the Common French records in this "fr" Description file.

...as expected due to close match with TOTAL DESCRIPTION Counts

...

all x prefixes added in, plus PreProduction naming convention applied, as expected. 

...

2 new records added/updated for 20230607, 

10 records (from all different effectiveTimes) removed

...

in Summary Component Stats report for this cycle (see link above)

...

6860 new records added/updated for 20230607, 

11613 records (from all different effectiveTimes) removed

...as expected ????????????????

10753 records have been added here for the Common French descriptions (11000241103) - question is why?!!

The snapshot of this file had only 20 records in it for the December 2021 release!

However when I run the June 2022 release, it now has over 10000 records!  (there’s only been 200 odd Description changes in this cycle!)

...

To be fair, looking at the Common French Description file (https://prod-release.ihtsdotools.org/api/centers/ch/products/snomed_ct_switzerland_releases[…]s/xsct2_Description_Snapshot-fr_CH1000195_20220607.txt), it looks like there were about 10254 records inactivated in the December release - however these were Born Inactive (in the first time release) and so I wouldn’t expect the system to create new attributeValue records for these?  But perhaps it has to if it can’t find existing attributeValue records for the old inactive records??

CHECKING WITH QUYEN...Quyen thinks it's a snowstorm bug, so I've raised ticket:

Jira
serverIHTSDO JIRA
serverIdb202d822-d767-33be-b234-fec5accd5d8c
keyISRS-1254

Michael confirmed them to be all valid in ISRS-1254

...

27928 new records added/updated for 20230607 - ***** FOR REFSETID: 2041000195100 *****

1563 records (from all different effectiveTimes) removed

..NOT as expected due to Summary Component Stats report for this cycle (see link above) showing:

  • 27928 new records added/updated for 20230607, 

  • 0 records inactivated

IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

DOES THIS MEAN THE SCS REPORT WILL BE INVALID, AS WILL SHOW NOW INACTIVATIONS BECAUSE IT'S A "BRAND NEW RELEASE" WITH NO "PREVIOUSPUBLISHEDPACKAGE" TO COMPARE THE CONTENT TO????

...

1292 new records added/updated for 20230607 - ***** FOR REFSETID: 2031000195108 *****

1311 records (from all different effectiveTimes) removed

...as expected ????????????????

 due to exact match in Summary Component Stats report for this cycle (see link above)

...

96 new records added/updated for 20230607 - ***** FOR REFSETID: 900000000000508004 + 900000000000509007 *****

86 records (from all different effectiveTimes) removed

...as expected ????????????????

...

0 records added/updated, as expected as this is Common French Language file??????

(but then shouldn't we be moving some into fr-ch, or were these already in December - no they weren't there, they requested no preferences for Common French, just Swiss French)

SUMMARY REPORT EXPECTING 11 - this looks like a problem with the Summary Component Report - possibly just because we're comparing to a DIFFERENT previous published package though, because we've generated the new one?!! 

...

161,454 new records added/updated for 20230607 - ***** FOR REFSETID: 2021000195106 *****

98,834 records (from all different effectiveTimes) removed

...as expected ????????????????

 due to exact match in Summary Component Stats report for this cycle (see link above)

...

25 new records added/updated for 20230607, 

IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

NO - RORY CONFIRMED (during our call at 17:00 on ) THAT THE 7478 RECORDS STILL LEFT IN THE CF module (11000241103) WERE JUST A MISTAKE WHEN HE WAS DOING THE ORIGINAL UPLOADS

Description (fr-ch) files158774

158,602 new records added/updated for 20230607 - 

147 records (from all different effectiveTimes) removed

So my manual validation shows:

  1. 147 records removed from original file   (all in SWISS module (2011000195101))
  2. 158,623 records added back in                 (all in SWISS module (2011000195101))
    1. 21 inactive records (ALL OF THEM) from original Description file had new DescriptionID's assigned to them, ***BUT WERE THEN ADDED BACK IN AS BORN-INACTIVE RECORDS***
      1. ?????????? as expected ????????????

      2. YES - RORY CONFIRMED (during our call at 17:00 on ) THAT these born inactive records are expected

    2. 125 active records were removed from the original Description file
      1. These were all in the Swiss Module (2011000195101), so not sure why we removed these, other than to just wipe the slate clean and start again??
      2. GOOD NEWS is that I spot checked several dozen and they're all JUST DescriptionID REFRESHES, as they all exist in the new content being added back in, so all good as we'd expect the ID's to be refreshed due to this being a "Brand new release"
    3. 158,623 records added in!!! (all with 20230607 effectiveTime, all in SWISS module (2011000195101) 
      1. All in the Swiss Module (2011000195101) as expected
      2. 21 of these records were born-inactive "0" status records
      3. From dozens of spot checks it looks like the 125 original records were all replaced (just had UUID's refreshed), so we're looking at 158,477 NEW records from the Common French import.
      4. PLUS figures match exactly to the "Desc by Lang" tab of the SCS report above:
        1. 21 "New Inactive"
        2. 158,602 "New" (125 original active records + 158,477 brand new records) 

...as expected due to close match with TOTAL DESCRIPTION Counts in Summary Component Stats report for this cycle (see link above)

IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

NO - RORY CONFIRMED (during our call at 17:00 on ) THAT THE 7478 RECORDS STILL LEFT IN THE CF module (11000241103) in the description-fr file WERE JUST A MISTAKE WHEN HE WAS DOING THE ORIGINAL UPLOADS.  They therefore now need to be added back into this description-fr-ch file in the Swiss module (2011000195101).

Description (it-ch) files2637

1326 new records added/updated for 20230607 - 

1311 records (from all different effectiveTimes) removed

So my manual validation shows:

  1. 1311 records removed from original file   (all in SWISS module (2011000195101))
  2. 1326 records added back in                          (all in SWISS module (2011000195101))
    1. 32 inactive records (ALL OF THEM) from original Description file had new DescriptionID's assigned to them
    2. 1279 active records were removed from the original Description file
      1. These were all in the Swiss Module (2011000195101), so not sure why we removed these, other than to just wipe the slate clean and start again??
      2. GOOD NEWS is that I spot checked several dozen and they're all JUST DescriptionID REFRESHES, as they all exist in the new content being added back in, so all good as we'd expect the ID's to be refreshed due to this being a "Brand new release"
    3. 40 records added in!!! (all with 20230607 effectiveTime, all in SWISS module (2011000195101) 
      1. All in the Swiss Module (2011000195101) as expected
      2. 34 of these records were born-inactive "0" status records
      3. From dozens of spot checks it looks like the 1279 original active records were all replaced (just had UUID's refreshed), so we're looking at 13 NEW records from the IT lang.
      4. PLUS figures match exactly to the "Desc by Lang" tab of the SCS report above:
        1. 34 "New Inactive"
        2. 1292 "New" (1279 original active records + 13 brand new records) 

 due to close match with TOTAL DESCRIPTION Counts in Summary Component Stats report for this cycle (see link above)

IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

Yes, Rory confirmed this is correct during our call at 17:00 on  

Relationship files64

37 new records added/updated for 20230607 - 

27

...

records (from all different effectiveTimes) removed

...NOT as expected due to Summary Component Stats report for this cycle (see link above) showing:

...

  • 37 new records added/

...

0 records inactivated

IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

  • updated for 20230607, 

  • 0 records inactivated

...

Some of the records removed appear to be absolutely valid, as they are previously inactive records (and therefore there's no need to keep them as "born inactive" records) (eg)

...

  • 2026638020    20220607    0    2011000195101    394812008    394733009    0    116680003    900000000000011006    900000000000451002
  • 11000241126    20220607    0    2011000195101    11000241103    900000000000443000

...

  •    0  

...

  •  116680003    900000000000011006  

...

  •  900000000000451002

Many more appear to be valid as they're simply "removing" these records to be replaced with the same record just with new effectiveTimes (eg)

  • <

...

  • 33588028    20211207    1    2011000195101

...

  •    224877003    308912000    0    116680003    900000000000011006    900000000000451002
  • >

...

  • 33588028    20230607    1    2011000195101

...

  •    224877003    308912000    0    116680003    900000000000011006    900000000000451002

Yet more appear to be added in validly as brand new concepts for this authoring cycle (eg)

...

  • > 123431000195124    20230607    1    2011000195101  

...

  •  72281000195108    365996006    0    116680003    900000000000011006    900000000000451002
  • > 123441000195127

...

  •    20230607    1    2011000195101  

...

  •  72281000195108  

...

  •  225598002    1    363714003  

...

  •  900000000000011006  

...

  •  900000000000451002

...

  • > 123481000195123    20230607    1    2011000195101

...

  •    82291000195104    417319006    0    116680003    900000000000011006    900000000000451002
  • > 123501000195126

...

  •    20230607    1    2011000195101  

...

However, that leaves 1 active record that has just been removed for no apparent reason:

...

  •  62271000195100    364830008    1  

...

  •  363702006    900000000000011006  

...

DID RORY PERHAPS INTENTIONALLY MISS OUT ALL CONTENT FOR THIS MODULE (WHEN IMPORTING THE CONTENT INTO THE FRESH NEW CODESYSTEM), BECAUSE THIS IS THE COMMON FRENCH MODULE (11000241103 | SNOMED CT Common French translation module (core metadata concept) |) AND THEREFORE WE'RE NOW GOING FOR AN APPROACH THAT JUST IMPORTS THE COMMON FRENCH CONTENT STRAIGHT INTO THE SWISS MODULE INSTEAD AND THEREFORE THERE IS NO LONGER A DEPENDENCY THERE ON THE COMMON FRENCH MODULE, AND THUS NO REASON TO INCLUDE IT IN THE SWISS PACKAGE????

  •  900000000000451002

The good news here is that the only record related to the now unnecessary Common French module (11000241103) was already inactive, and therefore gets removed anyway:

  • < 11000241126    20220607    0    2011000195101    11000241103    900000000000443000    0    116680003    900000000000011006    900000000000451002

IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

Yes, Rory confirmed this is correct during our call at 17:00 on  

Relationship Concrete Values files0

0 records added/updated

...as expected due to exact match

...

28030 new records added/updated for 20230607 

1566 records (from all different effectiveTimes) removed

...as expected ????????????????

...

in Summary Component Stats report for this cycle (see link above)

...

OWL Expression files

...

46

...


25 new records added/updated for 20230607 - 

...

21 records (from all different effectiveTimes) removed

...as expected ???????????????

...

?

Some of the records removed appear to be absolutely valid, as they are previously inactive records (and therefore there's no need to keep them as "born inactive" records) (eg)

  • 1ed01c6c-26ea-4b1f-a59e-c79fd882fc51    20220607    0    2011000195101    733073007    394812008    SubClassOf(:394812008 :394733009)

Many more appear to be valid as they're simply "removing" these records to be replaced with the same record just with new effectiveTimes (eg)

  • < 07806988-6dec-40d9-b4b9-53b4cd6458bd    20221207    1    2011000195101    733073007    52281000195106    SubClassOf(:52281000195106 :229824005)
  • > 07806988-6dec-40d9-b4b9-53b4cd6458bd    20230607    1    2011000195101    733073007    52281000195106    SubClassOf(:52281000195106 :229824005)

The good news here is that one of the records related to the now unnecessary Common French module (11000241103) was already inactive, and therefore gets removed anyway:

  • < 11000241126    20220607    0    2011000195101    11000241103    900000000000443000    0    116680003    900000000000011006    900000000000451002

The other one relating to this module also gets removed, as expected:

  • < b7c3c7f7-1967-0917-6c22-db19ddd83319    20211207    1    11000241103    733073007    11000241103    SubClassOf(:11000241103 :900000000000443000)

Yet more appear to be added in validly as brand new concepts for this authoring cycle (eg)

  • > 28b8c71c-7f57-46d1-af19-9344a5f47bef    20230607    1    2011000195101    733073007    72281000195108    SubClassOf(:72281000195108 :365996006)
  • > 3a3153be-35f2-49c4-ab6e-acebbc172227    20230607    1    2011000195101    733073007    62291000195101    SubClassOf(:62291000195101 :229824005)
  • > 699639a9-0d0f-4b43-8cb0-bf78e56e8c26    20230607    1    2011000195101    733073007    62281000195103    SubClassOf(:62281000195103 :229824005)
  • > 781e5e06-349f-402b-93e4-c9e77b193836    20230607    1    2011000195101    733073007    82291000195104    SubClassOf(:82291000195104 :417319006)
  • > eb98a3ab-c77b-411c-ad04-2965d66da434    20230607    1    2011000195101    733073007    72291000195105    SubClassOf(:72291000195105 :365508006)
  • > fac07ccd-6653-43d7-bdd1-d51d9d586a58    20230607    1    2011000195101    733073007    62271000195100    SubClassOf(:62271000195100 ObjectIntersectionOf(:229824005 :371735007))

IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

Yes, Rory confirmed this is correct during our call at 17:00 on  

...

 due to close match with TOTAL DESCRIPTION Counts in Summary Component Stats report for this cycle (see link above)

...

7479 new records added/updated for 20230607 - 

177,783 records (from all different effectiveTimes) removed

...as expected ????????????????

 as planned in INFRA-8967 in order to correctly split out the Swiss French records into their own "fr-ch" Description file, leaving ONLY the Common French records in this "fr" Description file.

...as expected due to close match with TOTAL DESCRIPTION Counts in Summary Component Stats report for this cycle (see link above)

...

158,623 new records added/updated for 20230607 - 

151 records (from all different effectiveTimes) removed

...as expected ????????????????

...as expected due to close match with TOTAL DESCRIPTION Counts in Summary Component Stats report for this cycle (see link above)

...

1326 new records added/updated for 20230607 - 

1311 records (from all different effectiveTimes) removed

...as expected ????????????????

 due to close match with TOTAL DESCRIPTION Counts in Summary Component Stats report for this cycle (see link above)

...

37 new records added/updated for 20230607 - 

27 records (from all different effectiveTimes) removed

...NOT as expected due to Summary Component Stats report for this cycle (see link above) showing:

  • 37 new records added/updated for 20230607, 

  • 0 records inactivated

IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

...

0 records added/updated

...as expected due to exact match in Summary Component Stats report for this cycle (see link above)

...

25 new records added/updated for 20230607 - 

21 records (from all different effectiveTimes) removed

...as expected ????????????????

...

TextDefinition (de-ch) files0

0 new records added/updated for 20230607 - 

0 records (from all different effectiveTimes) removed

...as expected

...

as this file was empty in the previous releases.

TextDefinition (en) files0

0 new records added/updated for 20230607 - 

0 records (from all different effectiveTimes) removed

...as expected

...

as this file was empty in the previous releases.

TextDefinition (fr) files1

0 new records added/updated for 20230607 - 

1 records (from all different effectiveTimes) removed

...

1 record related to the now unnecessary Common French module (11000241103) was removed as expected:

1020231000241112    20220331    1    11000241103    77386006    fr    900000000000550004    Patiente enceinte actuellement    900000000000017005

IS THIS TO BE EXPECTED BECAUSE WE'VE "STARTED FROM SCRATCH" IN THIS RELEASE??????

Yes, Rory confirmed this is correct during our call at 17:00 on  

TextDefinition (fr-ch) files0

0 new records added/updated for 20230607 - 

0 records (from all different effectiveTimes) removed

...as expected

...

as this file was empty in the previous releases.

TextDefinition (it-ch) files0

0 new records added/updated for 20230607 - 

0 records (from all different effectiveTimes) removed

...as expected

...

as this file was empty in the previous releases.