February 2023 PRODUCTION (PUBLISHED) to March 2023 PRODUCTION (Version 1 POST VERSIONING) traceability
SUMMARY COMPONENT REPORT (RUN BY DEVOPS POST-VERSIONING): https://docs.google.com/spreadsheets/d/1mayFvPxckC54AnuMEQQN23aDQrg8jrogUYPxuPvgRlA/edit#gid=0
SEP + LATERALIZABLE REFSET REPORT (RUN BY MARIA PRE-VERSIONING): https://docs.google.com/spreadsheets/d/1DZRz82bS9ZFw_IvdUm9Q-SUdAHARYonPD3uWq5UCfl0/edit#gid=0
Differences found in package Comparison | Number of RF2 records impacted | Related JIRA ticket(s) | Rationale |
---|---|---|---|
RefsetDescriptor files | 0 | 0 records added/updated as expected | |
Association Reference files |
1600 |
1158 records added/updated, plus |
442 records inactivated ...as expected as the totals ALMOST match exactly with Summary Component Stats report, but this report was run BEFORE we manually added in the 15 new records as part of the fixes in ISRS-1271 (spot checked these manually and they are present in new Association Snapshot + Full files as planned), so this brings us much closer to the figures in Comparison Diff files. In order to get an accurate comparison, we have to ALSO INCLUDE the SEP refset (734138000 + 734139008) totals from the RF2 files (manually tallied from the new Association Snapshot files) and add them to the "Hist Assoc" tab in the Summary Component Report: SEP (from RF2 diff files) = 36 changes: |
|
|
|
|
https://docs.google.com/spreadsheets/d/1-u-8pAsQaI2fkER4XbK4ogtxjc65rPnM0NcYZZAAeYk/edit#gid=0
This then, adds to the "Association" totals in the final refsets tab, in the Summary report stats: |
|
Makes a total of: |
|
Which matches exactly with the counts I got above from the RF2 files!! |
Attribute Value files |
3064 |
3000 records added/updated, plus |
64 inactivated ...as expected as the totals match exactly with Summary Component Stats report |
Language refset files |
8291 |
6914 records added/updated, plus |
1377 records inactivated *NOTE - NEED TO SEARCH THE LANG REFSET FILES FOR BOTH the following modules in order to get the full totals:
Confirmed as matches exactly with data in Summary Component Stats report - see Report linked above -
|
ExtendedMap files |
762 |
383 records added/updated, plus |
379 records inactivated Which matches exactly with the new detailed Map Reports WCI sent off the back of Maria's Map tasks, sent in an email from noreply@snomed.org entitled "Upload Successful", at |
10: |
50 on |
:
|
|
"inactivated" counts from diff
|
|
|
Simple Refset files |
37 |
28 records added/updated, plus |
9 records inactivated
|
|
SimpleMap files |
859 |
838 records added/updated, plus |
21 records inactivated Confirmed as matches with:
|
|
|
|
|
|
|
|
|
|
3
3 records updated, as expected for February 2023 effectiveTimes
February 2023 dates updated + Beta "x" prefixes removed, as expected
1565 records added/updated, plus
273 records inactivated
Confirmed as matches exactly with data in Summary Component Stats report - see Report linked above - ("added/updated" counts above match against total of "New" + "Changed DefnStatus" + "Reactivated" columns in report)
*NOTE - NEED TO SEARCH THE LANG REFSET FILES FOR BOTH the following modules in order to get the full totals:
- 20230228\t1\t900000000000207008
- 20230228\t1\t900000000000012004
3951 records added/updated, plus
555 records inactivated
Confirmed as matches exactly with data in Summary Component Stats report - see Report linked above - ("added/updated" counts above match against total of "New" + "Changed DefnStatus" + "Reactivated" columns in report)
NOTE: Updated total DOES NOT ALWAYS MATCH EXACTLY (like the others) - the report states 1255 inactivations instead of 1256. This is explained by the inclusion of the "<" character (which I use to search the Diff files to separate out the different change types) in many actual Terms. So searching for "< " instead (including the tab after <) finds instead 334 records updated, which is much closer.
*NOTE - NEED TO SEARCH THE LANG REFSET FILES FOR BOTH the following modules in order to get the full totals:
- 20230228\t1\t900000000000207008
- 20230228\t1\t900000000000012004
***** EXCEPT THIS TIME IT DOESN'T MATCH WITH THE REPORTS!!! IT'S 7 LESS THAN THE REPORTS, WHICH TALLIES WITH THE ISSUES RICK FOUND WHEN TRYING TO VERIFY THE FIGURES BEFORE FNIALISING THE MAPS!!! CONFIRMED on INFRA-10658... that both WCI + our internal Dev team are confident that the RF2 files are correct. The MARCH 2023 Release can therefore proceed as planned. The discrepancy is either between the snowstorm authoring vs browser branches, or between the snowstorm authoring branch vs RF2 files. Likely as not, the problem happened right at the start of our transition to FRI (Sept 21) when our old process was:
So it's likely that this last step was accidentally missed, resulting in the original active map records being in the RF2 files but NOT in the snowstorm authoring branch. Then, when they were inactivated in this cycle, the snowstorm branch had no visibility of the original active records, and therefore the inactivations couldn't happen. This is because the new process is:
Therefore snowstorm can't inactivate records that aren't active (or even existing) in the branch, and the SRS can't inactivate the active records it DOES have as it's no longer doing the reconciliation, instead it's relying on snowstorm to pass it delta records for the inactivations. This will be proven, and full reconciliations run between all systems to identify the gaps, change any code required, and apply content fixes for al gaps identified... INFRA-10658 - final agreed plan is to wait until the ICDO release is published, and then the dev ops team will import the Snapshot into the authoring platform, and that should get everything up to date. Therefore no changes required for this March International release. | |||
ModuleDependency files | 3 | 3 records updated, as expected for March 2023 effectiveTimes | |
Readme file | March 2023 dates updated + Beta "x" prefixes removed, as expected | ||
Concept files | 1488 | 1063 records added/updated, plus 425 records |
11304 records added/updated, plus
7898 recordsinactivated Confirmed as matches exactly with data in Summary Component Stats report - see Report linked above - ("added/updated" counts above match against total of "New" + "Changed DefnStatus" + "Reactivated" columns in report) |
*NOTE - NEED TO SEARCH THE LANG REFSET FILES FOR BOTH the following modules in order to get the full totals:
|
Description files |
3854 |
3220 records added/updated, plus |
634 records inactivated Confirmed as matches exactly with data in Summary Component Stats report - see Report linked above -
|
3778 records added/updated, plus
275 records inactivated
Confirmed as matches exactly with data in Summary Component Stats report - see Report linked above - ("added/updated" counts above match
|
|
|
NOTE |
: Updated total DOES NOT ALWAYS MATCH EXACTLY (like the others) - the report states 1255 inactivations instead of 1256. This is explained by the inclusion of the "<" character (which I use to search the Diff files to separate out the different change types) in many actual Terms. So searching for "< " instead (including the tab after <) finds instead 334 records updated, which is much closer. *NOTE - NEED TO SEARCH THE LANG REFSET FILES FOR BOTH the following modules in order to get the full totals:
| |||
Relationship files | 15159 | 8959 |
- 20230228\t1\t900000000000207008
- 20230228\t1\t900000000000012004
0 records updated, plus
0 records inactivated, plus
9000000000004970000 records added
As expected due to full migration to OWLExpression file in the July 2019 International Edition
records added/updated, plus |
6200 records inactivated Confirmed as matches exactly with data in Summary Component Stats report - see Report linked above - ("added/updated" counts above match against total of "New" + "Changed DefnStatus" + "Reactivated" columns in report) *NOTE |
- NEED TO SEARCH THE LANG REFSET FILES FOR BOTH the following modules in order to get the full totals:
| |||
Relationship Concrete Values files | 147 | 121 records added/updated, plus 26 records inactivated Confirmed as matches exactly with data in Summary Component Stats report - see Report linked above - ("added/updated" counts above match against total of "New" + "Changed DefnStatus" + "Reactivated" columns in report) | |
OwlExpression (Axiom) files | 2912 | 2486 records added/updated, plus 426 records inactivated Confirmed as matches exactly with data in Summary Component Stats report - see Report linked above - ("added/updated" counts above match against total of "New" + "Changed DefnStatus" + "Reactivated" columns in report) *NOTE - NEED TO SEARCH THE LANG REFSET FILES FOR BOTH the following modules in order to get the full totals:
| |
Stated Relationship files | 0 | 0 records updated, plus 0 records inactivated, plus 9000000000004970000 records added As expected due to full migration to OWLExpression file in the July 2019 International Edition | |
TextDefinition files | 181 | 54 records added/updated, plus 127 records inactivated Confirmed as matches exactly with data in Summary Component Stats report - see Report linked above - ("added/updated" counts above match against total of "New" + "Changed DefnStatus" + "Reactivated" columns in report) NOTE: Updated total DOES NOT ALWAYS MATCH EXACTLY (like the others) - the report states 1255 inactivations instead of 1256. This is explained by the inclusion of the "<" character (which I use to search the Diff files to separate out the different change types) in many actual Terms. So searching for "< " instead (including the tab after <) finds instead 334 records updated, which is much closer. | |
MRCM AttributeDomain files | 0 | 0 records added/updated as expected | |
MRCM AttributeRange files | 0 | 0 records added/updated as expected | |
MRCM Domain files | 0 | 0 records added/updated as expected |
0 records added/updated as expected
1 records added/updated as expected
< e25ae83a-4da4-4c87-b254-f5399dd9b088 20180131 1 900000000000012004 723562003 719722006 << 719982003 |Process (qualifier value)| (<< 363787002 |Observable entity (observable entity)| OR << 404684003 |Clinical finding (finding)|): [0..*] { [0..1] 719722006 |Has realization| = << 719982003 |Process (qualifier value)| } 723597001 723596005
---
> e25ae83a-4da4-4c87-b254-f5399dd9b088 20230228 1 900000000000012004 723562003 719722006 << 272379006 |Event (event)| OR << 404684003 |Clinical finding (finding)| OR << 71388002 |Procedure (procedure)| OR << 719982003 |Process (qualifier value)| (<< 363787002 |Observable entity (observable entity)| OR << 404684003 |Clinical finding (finding)|): [0..*] { [0..1] 719722006 |Has realization| = (<< 272379006 |Event (event)| OR << 404684003 |Clinical finding (finding)| OR << 71388002 |Procedure (procedure)| OR << 719982003 |Process (qualifier value)|) } 723597001 723596005
As confirmed by Maria + Yong due to MRCM changes intentionally made by Yong this cycle...
*NOTE - NEED TO SEARCH THE LANG REFSET FILES FOR BOTH the following modules in order to get the full totals:
- 20230228\t1\t900000000000207008
- 20230228\t1\t900000000000012004
1 records added/updated as expected
As confirmed by Maria + Yong due to MRCM changes intentionally made by Yong this cycle...
*NOTE - NEED TO SEARCH THE LANG REFSET FILES FOR BOTH the following modules in order to get the full totals:
MRCM ModuleScope files | 0 | 0 records added/updated as expected |
February 2023 PRODUCTION (PUBLISHED) to March 2023 PRODUCTION (Version 1 PRE VERSIONING) traceability
SUMMARY COMPONENT REPORT (RUN BY MARIA PRE-VERSIONING): https://docs.google.com/spreadsheets/d/1A2EAHWQbZBUHiXQKddiaeSzL-kJbwQohwFdLLruK0RI/edit#gid=0
SEP + LATERALIZABLE REFSET REPORT (RUN BY MARIA PRE-VERSIONING): https://docs.google.com/spreadsheets/d/1DZRz82bS9ZFw_IvdUm9Q-SUdAHARYonPD3uWq5UCfl0/edit#gid=0
Differences found in package Comparison | Number of RF2 records impacted | Related JIRA ticket(s) | Rationale |
---|---|---|---|
RefsetDescriptor files | 0 | 0 records added/updated as expected | |
Association Reference files | 1600 | 1158 records added/updated, plus 442 records inactivated ...as expected as the totals ALMOST match exactly with Summary Component Stats report, but you also need to incorporate SEP changes: In order to get an accurate comparison, we have to ALSO INCLUDE the SEP refset (734138000 + 734139008) totals from the RF2 files (manually tallied from the new Association Snapshot files) and add them to the "Hist Assoc" tab in the Summary Component Report: SEP (from RF2 diff files) = 36 changes:
This then, adds to the "Association" totals in the final refsets tab, in the Summary report stats:
Makes a total of:
Which matches exactly with the counts I got above from the RF2 files!! | |
Attribute Value files | 3000 | 3000 records added/updated, plus 64 inactivated ...as expected as the totals match exactly with Summary Component Stats report | |
Language refset files | 8291 | 6914 records added/updated, plus 1377 records inactivated Confirmed as matches exactly with data in Summary Component Stats report - see Report linked above - ("added/updated" counts above match against total of "New" + "Changed DefnStatus" + "Reactivated" columns in report) | |
ExtendedMap files | 762 | 383 records added/updated, plus 379 records inactivated Which matches exactly with the new detailed Map Reports WCI sent off the back of Maria's Map tasks, sent in an email from noreply@snomed.org entitled "Upload Successful", at 10:50 on :
***** EXCEPT THIS TIME IT DOESN'T MATCH WITH THE REPORTS!!! This was proven to be because the "Total Changed mapped entries (modified) this release" field in the report was populated (with 16) for the first time since it was first added to these reports. This is to catch a rare situation where the mapping team updates the Map Advice, but does NOT change the Map Source or Target. Therefore this value needs to be included in my total counts every release cycle, otherwise we run the risk of not reconciling. Adding it now to the totals to add up from the report. | |
Simple Refset files | 37 | 28 records added/updated, plus 9 records inactivated
| |
SimpleMap files | 859 | 838 records added/updated, plplus 21 records inactivated Confirmed as matches with:
***** EXCEPT THIS TIME IT DOESN'T MATCH WITH THE REPORTS!!! IT'S 7 LESS THAN THE REPORTS, WHICH TALLIES WITH THE ISSUES RICK FOUND WHEN TRYING TO VERIFY THE FIGURES BEFORE FNIALISING THE MAPS!!! INVESTIGATING ..on INFRA-10658... | |
ModuleDependency files | 3 | 3 records updated, as expected for March 2023 effectiveTimes | |
Readme file | March 2023 dates updated + Beta "x" prefixes added, as expected | ||
Concept files | 1488 | 1063 records added/updated, plus 425 records inactivated Confirmed as matches exactly with data in Summary Component Stats report - see Report linked above - ("added/updated" counts above match against total of "New" + "Changed DefnStatus" + "Reactivated" columns in report) | |
Description files | 3854 | 3220 records added/updated, plus 634 records inactivated Confirmed as matches exactly with data in Summary Component Stats report - see Report linked above - ("added/updated" counts above match against total of "New" + "Changed DefnStatus" + "Reactivated" columns in report) NOTE: Updated total DOES NOT ALWAYS MATCH EXACTLY (like the others) - the report states 1255 inactivations instead of 1256. This is explained by the inclusion of the "<" character (which I use to search the Diff files to separate out the different change types) in many actual Terms. So searching for "< " instead (including the tab after <) finds instead 334 records updated, which is much closer. | |
Relationship files | 15159 | 8959 records added/updated, plus 6200 records inactivated Confirmed as matches exactly with data in Summary Component Stats report - see Report linked above - ("added/updated" counts above match against total of "New" + "Changed DefnStatus" + "Reactivated" columns in report) | |
Relationship Concrete Values files | 147 | 121 records added/updated, plus 26 records inactivated Confirmed as matches exactly with data in Summary Component Stats report - see Report linked above - ("added/updated" counts above match against total of "New" + "Changed DefnStatus" + "Reactivated" columns in report) | |
OwlExpression (Axiom) files | 2912 | 2486 records added/updated, plus 426 records inactivated Confirmed as matches exactly with data in Summary Component Stats report - see Report linked above - ("added/updated" counts above match against total of "New" + "Changed DefnStatus" + "Reactivated" columns in report) | |
Stated Relationship files | 0 | 0 records updated, plus 0 records inactivated, plus 9000000000004970000 records added As expected due to full migration to OWLExpression file in the July 2019 International Edition | |
TextDefinition files | 181 | 54 records added/updated, plus 127 records inactivated Confirmed as matches exactly with data in Summary Component Stats report - see Report linked above - ("added/updated" counts above match against total of "New" + "Changed DefnStatus" + "Reactivated" columns in report) NOTE: Updated total DOES NOT ALWAYS MATCH EXACTLY (like the others) - the report states 1255 inactivations instead of 1256. This is explained by the inclusion of the "<" character (which I use to search the Diff files to separate out the different change types) in many actual Terms. So searching for "< " instead (including the tab after <) finds instead 334 records updated, which is much closer. | |
MRCM AttributeDomain files | 0 | 0 records added/updated as expected | |
MRCM AttributeRange files | 0 | 0 records added/updated as expected | |
MRCM Domain files | 0 | 0 records added/updated as expected | |
MRCM ModuleScope files | 0 | 0 records added/updated as expected |