TECHNICAL REVIEW
| Issue | Resolution actions |
|---|---|
Born inactive stated relationships found for PreAlpha20170131 release RESOLUTION: Relevant born inactive relationships deleted. | MCH - These born inactive concepts and stated relationships were not identified during authoring validation due to we don't run these type of assertions as they compare against previous release package which slows down the validation process. Maybe we should consider to update existing assertions so that they can be run at task/project level validations as well. These assertions are run during the daily release build. AAT - we therefore asked Monica/Donna whether or not we have scope to have the RVF slightly in order to cover these issues as part of the authoring task validation. Monica confirmed this would not be possible if these do slow down the validation process.
|
Born inactive concepts found for preAlpha 20170131 release 717739000 20170131 0 900000000000207008 900000000000074008 RESOLUTION: Relevant born inactive concepts deleted. | Same as above |
The following new RVF failure appeared after applying the Alpha feedback fixes that the Content Team implemented: testCategory: "release-type-validation", RESOLUTION: 12 Relationships deleted completely, as they were new to the 20170131 Release cycle, and therefore not previously published. | Same as above |
The following RVF failure was newly added after the Alpha fixes that the Content Team implemented: testCategory: "release-type-validation", RESOLUTION: Concept 723067001 deleted completely, as it was new to the 20170131 Release cycle, and therefore not previously published. | Same as above |
Six concepts are in the 'SNOMED CT model component module (core metadata concept)' module but the relationships in question here are in the 'SNOMED CT core module (core metadata concept)' module. The associated descriptions are all in the same module as the concept entries. 718291003 RESOLUTION: 5 Inferred relationships have been inactivated, and 5 new ones activated with the correct module ID, in order to align the Concept moduleID with the related Relationship moduleID's. The final concept (900000000000441003) is the Model Component concept, and therefore has its Relationship in the Core module and not the Model Component module for a valid reason, as follows: • The 2 root modules (Core and Model Component) were setup in the first place so that the Core was dependent on the Model Component module The only action taken for this concept, therefore, was to update the comments in the code to ensure that this valid state is retained going forward. | This was a defect in the classifier wrapper, which MCH has now resolved. |
ISRS-103, ISRS-104, ISRS-106, ISRS-107, ISRS-108, ISRS-109 and ISRS-110 | These are all to be ignored as warnings - do we need to whitelist these? (especially as we've now open-sourced the RVF) Some of these can be fixed by whitelisting, but some are restraints in the assertions - so we need to fix them. The whitelisting framework is still a reasonably long way away due to capacity - most likely Q3/Q4 2017. |
| The Release Feedback process is labour intensive and could be refined |
|
CONTENT REVIEW
| Issue | Resolution actions |
|---|---|
Three retired concepts have a historical relationship of an inappropriate type: RESOLUTION: IHTSDO Content Team resolved as follows: 139521000: Tingling: [sensation] or [has sensation] (finding) now retired as ambiguous The third concept (700043003) was found to be valid, and therefore no action required | Simply a genuine human error, so the content team will improve the editorial guidance in order to prevent these issues going forward. |
Ten RF1 descriptions for active concepts are status 10 (MOVED_TO), alongside descriptions with active status: RESOLUTION: IHTSDO Content team have inactivated all of the descriptions in question with reason "moved elsewhere" | The Task originally created to resolve this issue somehow got deleted as part of the process, and so never made it into the Jan 2017 Release.
|
Three concepts in RF1 have more than one historical relationship (except MAY_BE_As, MOVED_TOs OR WAS_As): RESOLUTION: IHTSDO Content Team remodelled so that the three concepts now have one reason for inactivation and one historical relationship each as appropriate. | Simply a genuine human error, so the content team will improve the editorial guidance in order to prevent these issues going forward. |
As part of the previous editing cycle, a minor issue was found with several relationships, which had targets with different semantic tags to their sources. In discussion with the Content Team at the time, we agreed to fix all but one, which was seen to be an example of the exception to the rule, and therefore a valid relationship to retain: id effectiveTime active moduleId sourceId destinationId relationshipGroup typeId characteristicTypeId modifierId RULEID SCTID DETAILS TERMLIST RESOLUTION: IHTSDO Content Team remodelled this concept and removed the disorder parent. | Simply a genuine human error, so the content team will improve the editorial guidance in order to prevent these issues going forward. |
The following 3 concepts had the Core module assigned to them during authoring, instead of the Concept Model module: 719367001 RESOLUTION: 3 records updated to the correct ModuleID in the Concept Delta file in the Beta release | These were not assigned, the tool defaulted to the incorrect module ID even when a concept was cloned. Core module vs Model component. This should be default as per hierarchy selected. This was a new item introduced by SCA. Previously it was not a choice for an author to make. Now the team are aware and guidance has been written into the Ed. guide.
|
| ISRS-90 The concept 722541007 has concept modeling of Causative agent (attribute) = Infectious process (qualifier value). The value "infectious process" should not be allowed because it is not in the range for causative agent. RESOLUTION: Relationship record deleted, to remove the erroneous attribute Causative agent (attribute) | This was identified by the new MRCM validation, and therefore the long term solution for this is to get the MRCM rules into both the SCA input validation, and the RVF, so that these issues can be resolved as part of the editing cycle going forward. |
The concept 723017008 has After (attribute) = Intraocular lens implant (physical object). This violated the range for After attribute that should only take values from clinical finding hierarchy or procedure hierarchy. RESOLUTION: Relationship record updated, to change the erroneous attribute Intraocular lens implant (physical object) (313236002), and add in the new attribute 69724002|Insertion of prosthetic intraocular lens (procedure)| | MRCM rules ought to pick this up or not allow improper selection. Batch uploads bypass the MRCM rules therefore these can slip through. Since this occurred, the validation of ICD-11 has been changed from small amounts of spot checks to thorough checking by the ICD-11 authors. |
A new concept - 723067001 | Transcatheter aortic valve replacement (procedure) | - is a duplicate and so has been inactivated within the same authoring cycle. RESOLUTION: Concept deleted completely, as it was new to the 20170131 Release cycle, and therefore not previously published. | This was not a duplicate in fact but in as much as it was questioned by an member from a clinical perspective so the right thing to do was to pull it back till further investigation of meaning was carried out. Therefore this was a one off and will not re-occur. |
ADHA ran the alpha package through their validation routines. Technically, it passed – from a content persective there looks like a number of concept model violations.. · AFTER(255234002) - only modelled with valid destination concepts RESOLUTION: 1 Relationship deleted completely, as it was new to the 20170131 Release cycle, and therefore not previously published. | Same as ISRS 91: Do these ADHA-discovered issues need to be covered in our input validation and/or the RVF? |
ADHA ran the alpha package through their validation routines. Technically, it passed – from a content persective there looks like a number of concept model violations.. · CAUSATIVE AGENT(246075003) - only modelled with valid destination concepts RESOLUTION: 1 Relationship record updated, in order to replace the Causative agent attribute with Pathological process. | Same as ISRS 90: Do these ADHA-discovered issues need to be covered in our input validation and/or the RVF? |
ADHA ran the alpha package through their validation routines. Technically, it passed – from a content persective there looks like a number of concept model violations... · HAS INTERPRETATION(363713009) - only modelled with valid destination concepts RESOLUTION: 3 records removed, plus 3 records added in the Relationship and Stated Relationship Delta files in the Beta release, in order to change the attribute to "Interprets". | Do these ADHA-discovered issues need to be covered in our input validation and/or the RVF? ICD-11 batch allows incorrect attributes to be selected. |
ADHA ran the alpha package through their validation routines. Technically, it passed – from a content persective there looks like a number of concept model violations.. · OCCURRENCE(246454002) - only modelled with valid destination concepts RESOLUTION: 2 Relationships deleted completely, as they were new to the 20170131 Release cycle, and therefore not previously published. | Do these ADHA-discovered issues need to be covered in our input validation and/or the RVF? ICD-11 batch allows incorrect attributes to be selected. This is partly due to the fact that an allowable value for this attribute is CONGENITAL so it is easy to presume that ACQUIRED would belong here too. Guidance has been clarified in the Ed. guide. |
ADHA ran the alpha package through their validation routines. Technically, it passed – from a content perspective there looks like a number of concept model violations.. I’ve got almost 700 observable entity concepts failing the tests below… but… · observable entity(363787002) only approved defining attributes used RESOLVED: This was traced back to a change that was made to the Concept model for Observables, that was implemented in September 2016. The editorial guide was updated accordingly, but has not been published since the update was made, and so ADHA did not have visibility of the updates. IHTSDO therefore provided them with the information on the new Concept model, and they will update their validation rules accordingly. This is therefore a non-issue and no action is required. | Do these ADHA-discovered issues need to be covered in our input validation and/or the RVF? Publish the updated editorial guidance along with the Alpha release. |
The following RVF failure is now reporting 7 Active Descriptions that have Inactivation reasons assigned to them: assertionText: "Active descriptions do not have active inactivation indicators", Either the Descriptions need to be inactivated accordingly, or the reasons updated/removed as the Descriptions should remain Active. RESOLUTION: 6 Inactivation reasons were inactivated (as they were previously published), and 1 was deleted (as it was new to this release cycle). |
|
The new refset being added to the January 2017 International Edition (723264001 - Lateralizable body structure reference set (foundation metadata concept)) needs a new record added to the International edition refsetDescriptor files. RESOLUTION: The following refsetDescriptor record has been added accordingly: bbdcc9fc-5bc6-415a-a5ee-c5adfc569bd5 20170131 1 900000000000207008 900000000000456007 723264001 449608002 900000000000461009 0 | This kind of issue is now being addressed by the new Service Acceptance process, which will prevent these kind of pre-dependencies from being missed at the start of projects. |
5251000119108 |Acquired arteriovenous malformation (disorder)| incorrectly uses 255396000 |Acquired (qualifier value)| as the value of |Occurrence| IHTSDO removed attribute value pair of Occurrence - Acquired. Concept therefore needs to be changed to primitive. The related concept, 402847002 |Acquired arteriovenous malformation of the skin (disorder)| was also identified to have an invalid attribute. RESOLUTION: Both concepts have been remodelled accordingly. | Same as ISRS-96 |
Three retired metadata concepts were erroneously moved into the Core module: CONCEPTID CONCEPTSTATUS FULLYSPECIFIEDNAME These metadata concepts were transferred into the core module when they became inactive, so the RF1 conversion tool thinks they're relevant. The RF1 conversion filters out concepts with a metadata semantic tag, but only if they're also in the model module. RESOLUTION: ModuleID for all three concepts assigned to the Model Component module (900000000000012004). | Simply a genuine human error, so the content team will improve the editorial guidance in order to prevent these issues going forward. |
There are 21 Historical Associations that are active, but which have been found to have inactive concepts for their targetComponentID: id effectivetime active moduleid refsetid referencedcomponentid targetcomponentid RESOLUTION: It was discovered that 2 of the above source concepts actually had 2 concepts referenced, and so 23 associationReference records were updated in order to replace the inactive targetComponentID's with the relevant active concepts. In addition to this, the system then automatically inactivated 12 additional associationReference records, where the source concept was also associated to "Limited components" ([900000000000486000|limited]): 95d6b955-adb9-59fc-a011-93ee592153b1 20170131 0 900000000000207008 900000000000527005 149532003 287840006 | This issue needs particular attention, as one of our members is asking why the fix for this changed history? The issue here is that when the targetComponentID has been changed in the past, the record was inactivated and re-created with the new targetComponentID. However, the termServer considered this a mutable field, and therefore since we moved to the termServer if we change the targetComponentID field it will simply update the record instead of inactivating and re-creating.
Both David and Linda agreed the following:
"As far as I can tell there is nothing in the documents that these fields to be immutable. However, there are some reasons that suggest to me it might be better for them to be immutable.
For example:
Arguably the only case where it would "reasonable" to treat the target as mutable would be where
A(inactive)=B(active)
Then B is inactivated and a new historical relationship is added:
B(inactive)=C(active)
in this case one could argue that the transitive result represents an update of the existing association.
A(inactive)=C(active)
However, overall I would suggest that immutability is clearer and safer in all cases. We therefore need to take a look at the termServer functionality and update it accordingly...
|
| Mapping Release Notes - They were questioned by our members due to some inaccuracies in the ICD10 map wording. |
|
Future Improvements
| Area for improvement | Actions |
|---|---|
Walk through known issues with the content team, to ensure they will all be resolved in time for the Jan 2018 release… | AAT to discuss with Lesley |
ICD-0 map can actually be maintained in the mapping tool now, (and has been able to do so for a year) , so we now need to move to that as termServer isn’t maintaining it properly | AAT to setup a call to discuss with Lesley + Donna + Yong + WCI... |
Some Namespace concepts are now being created with the requesting company/body in as a preferred term (eg) 1000198 but this is not being consistently applied (eg) 1000201 - so can we either do it consistently or not at all... | AAT to discuss with Lesley - Yohani actually does this, so AAT to speak to Yo instead... |
| We received reports in from one of the community to tell us that they couldn’t access some of the pages that the team linked into the Release Notes. | AAT to agree with Lesley to just open up content pages? |
MAPPING ISSUES:
See ticket https://jira.ihtsdotools.org/browse/ISRS-178
| AAT to discuss with Lesley + Donna - include in ICD-0 conversation |
Some Namespace concepts are now being created with the requesting company/body in as a preferred term (eg) 1000198 but this is not being consistently applied (eg) 1000201 - so can we either do it consistently or not at all... | AAT to discuss with Lesley - Yohani actually does this, so AAT to speak to Yo instead... company is available, so should be no issue to include: https://docs.google.com/spreadsheets/d/1qbCiwCcsW3O9MOsNw2oUbSO_BjNhm0VNq5NEiMqxMf4/edit?ts=59b006c6#gid=0 |
| We received reports in from one of the community to tell us that they couldn’t access some of the pages that the team linked into the Release Notes. | Lesley agreed to just open up content pages |
Authors are concerned with the fact that Authors are concerned with the fact that we can’t merge alpha/beta fixes back to main until end of July - this has always been the case but never a problem before due to small volumes of change in the alpha/beta period in the past. Going forward this will likely increase, so we need to find a way to merge changes back into main after each release stable phase (alpha/beta/member) without any risk to MAIN | Rory confirmed that we can’t promote to main every alpha/beta/member release because you can’t promote without re-basing - so the only option is to fix this restriction might be to have 2x fix branches, then each time you want to do a fix, you test it on the first (release fix) branch until you’re happy with it, then once testing signed off you replicate the change(s) in the second (merge fix) branch, which we will then promote to main in order to keep the authors in sync. this will allow the merge branch to get re_based with the latest content (for the next editing cycle), whilst keeping the release branch clean with only the current release’s content | in it…
MAPPING ISSUES:
See ticket https://jira.ihtsdotools.org/browse/ISRS-178
| AAT to discuss with Lesley + Donna |
in it… AAT to agree the update to the release process with Lesley/Maria in order to ensure she understands that she will need to make each fix twice in the next release... Lesley to talk to Maria/Monica first to get perspective on how much impact the delays had in the July 2017 release (Toni, Penni and Maria) - then we can all talk together about whether or not tp update the process... | |
UKTC MAPPING ISSUES: Both Hazel and Genevieve managed to have an adverse impact on our mapping process - so we need to refine the process to ensure that we can check both “New”/“Editing”/“Finished" statuses and “QA” work (Need to ask WCI to update the tool here?? Or just have a manual process whereby Donna asks them on the weekly calls to confirm they have nothing outstanding in QA status after content cut off for each release) AAT to discuss with Lesley + Donna | Lesley confirmed Content Release Managers should also have this as a check box on their list - both to communicate to the externals not to touch anythign during Release periods, and also to look into changing externals access rights in the Mapping Tool to Read-only durin htese periods... |
NEED TO DECIDE THE PRIORITY OF THE FIXES FOR THE MRCM INFERRED + STATED HISTORICAL ISSUES - for example, there was one concept (717703006) which Jim said should be fixed asap | AAT to discuss with Lesley + Jim - Lesley to plan this out |
BATCH IMPORT APPEARS TO BE POTENTIALLY RISKY - quite a few issues we came across in the July 2017 seemed to be rooted in batch content added - so would be good to review and refine the validation around Bulk imports - for example ISRS-221 (concepts marked in white in the spreadsheet attached) - all bulk imported by Toni (or so Monica thinks) | Not much we can do about this technically - need to discuss with Lesley to ensure everyone takes the utmost care when implementing Batch Imports - and also tries their best to finalise content before importing, rather than importing and then changing FSN's, etc. Batches are all different processes, so Lesley needs me to send her all of the batches in question in order to fix them. I sent her ISRS-221, and she said this was an issue with the dirty content from termMed - Lesley will discuss with Rory how to get these reviewed BEFORE they get put into a task... |
Some documentation gaps that still need closing - matt cordell to send details so we can close them in time for jan 2018 | AAT to speak to Maria when Matt sends the list |
| ISRS-221 - we need to agree what the relationship between inactivation indicator & historical association should be, and raise tickets to have the SCA and RVF enhanced to match | AAT to discuss with Michael/Hunter |
Re-basing issues! rfbintjula yet again got re-based against our express request | RDA confirmed that this ticket has already been raised to be able to lock the Release branches - it will definitely get done before the Jan 2018 Release cycle starts in November 2017. |
New potential clinical safety issue (see email from Monica at 14:30 on 13/7/2017 - any way to prevent this from happening again going forward? | Not really, but the main learning point here is that we need to ensure that we have reached a full (and unanimous) decision on whether or not these issues are actually Clinical Risks, before we raise the red flag with the Release/Technical teams and start off the whole recall process. If this had been discussed fully with all Content authors, we may have not needed to perform the recall after all. Therefore, we need an internal process first which goes through and confirms with their own stakeholders before raising the red flag. |
Need to work through all issues found in pre-alpha, alpha and beta testing, both internally +++++ externally, and ensure that we retrospectively update the rvf to cover all these scenarios | AAT to discuss with Hunter as part of the next cycle of Release improvements. |
|
|
We need to fix the conflicts that can happen when runnig multiple jenkins jobs, or possibly even jenkins jobs vs validation runs!
|
|
Need to discuss whether or not we should be going back to the uktc and telling them we will no longer consider any rf1 defects as part of the current release cycle - only as part of ongoing editing cycles for future releases? (only issue here is that i no longer have a friendly contact in the uktc, or even someone who will talk to us or answer any of our emails! so could be an issue getting the message across, as might sound harsh via email…) |
|
Traceability database logging - is there any way to break down the logs for concepts which have huge logs? example was isrs-230, where we couldn’t track the root cause of the issue because the log for just one of the 3 concepts in question was over 11mb!! the logging needs to cut off at 1mb otherwise it’ll break the indexes, so at the moment it just completely fails to log for any large complex changes! if we could break it down into smaller chunks then this’d be great, as given the likelihood of content authoring continuing to be complex going forward (with all the remodelling work coming down the pipeline), we otherwise run the risk of having holes in our audit trail, making it more time-consuming to track issues down. |
|
How realistic would it be to cut off the content slightly earlier than normal next release cycle? As we saw from the craziness in this release, the volume and complexity of the authoring changes meant we were working long hours right up until the very last day of the release! this is fine from my perspective, but it will impact the content release lead if this continues, plus it’s a huge risk to the release, as human error is far more likely to creep in if we’re working that late. | AAT to discuss this with the content team for the July 2018 release, as the next Jan 2018 release cycle is already confirmed. |