January 2018 International Edition release - Post Implementation Review

Walk through known issues with the content team, to ensure they will all be resolved in time for the Jan 2018 release…

ICD-0 map can actually be maintained in the mapping tool now, (and has been able to do so for a year) , so we now need to move to that as termServer isn’t maintaining it properly

MAPPING ISSUES:

1. 10 inactivated records came through in the termserver simplemap export:

                                          See ticket https://jira.ihtsdotools.org/browse/ISRS-178

1. Donna thought these were related to ICD-10 codes (instead of ICD-0 like it supposed to be the scope for simplemap, as ICD-10 is supposed to only be extendedmap), but Yong thought they were Anatomy codes, inactivated automatically by the system because the related concepts had been inactivated. Yong then confirmed these were outside of the scope of ICD-0 mapping.
2. We may have a small disconnect here, as I’m asking Donna to account for the content of simplemap and extendedmap files, yet she doesn’t have visibility of all of the input for these files!

Some Namespace concepts are now being created with the requesting company/body in as a preferred term (eg) 1000198   but this is not being consistently applied (eg) 1000201 - so can we either do it consistently or not at all...


Authors are concerned with the fact that we can’t merge alpha/beta fixes back to main until end of July - this has always been the case but never a problem before due to small volumes of change in the alpha/beta period in the past.  Going forward this will likely increase, so we need to find a way to merge changes back into main after each release stable phase (alpha/beta/member) without any risk to MAIN

UKTC MAPPING ISSUES:

Both Hazel and Genevieve managed to have an adverse impact on our mapping process - so we need to refine the process to ensure that we can check both “New”/“Editing”/“Finished" statuses and “QA” work (Need to ask WCI to update the tool here??  Or just have a manual process whereby Donna asks them on the weekly calls to confirm they have nothing outstanding in QA status after content cut off for each release)

NEED TO DECIDE THE PRIORITY OF THE FIXES FOR THE MRCM INFERRED + STATED HISTORICAL ISSUES - for example, there was one concept (717703006) which Jim said should be fixed asap

BATCH IMPORT APPEARS TO BE POTENTIALLY RISKY - quite a few issues we came across in the July 2017 seemed to be rooted in batch content added - so would be good to review and refine the validation around Bulk imports - for example ISRS-221 (concepts marked in white in the spreadsheet attached) - all bulk imported by Toni (or so Monica thinks)

Some documentation gaps that still need closing - matt cordell to send details so we can close them in time for jan 2018

Re-basing issues! rfbintjula yet again got re-based against our express request 

New potential clinical safety issue (see email from Monica at 14:30 on 13/7/2017 - any way to prevent this from happening again going forward?

Need to work through all issues found in pre-alpha, alpha and beta testing, both internally +++++ externally, and ensure that we retrospectively update the rvf to cover all these scenarios

1. We need to start working on moving the technical back end changes to be able to be fixed by the authors instead. This is causing us a lot of problems in the Alpha/Beta period, where backend fixes are clashing with front end fixes (eg) https://jira.ihtsdotools.org/browse/ISRS-150?filter=14082 - where back end fixes (PLUS in this case also changes to the were made for Yong’s SEP Refsets) and so an AssociationReference Delta created for the Release and placed int he Externally Maintained Refset folder, then continue to overwrite changes to the Historical Associations that Monica was trying to fix as part of the Beta feedback period.

1. This is only likely to get worse going forward, so the more we can move to the authors fixing everything in the SCA (and therefore delivering everything through the termServer Delta exports) the better
2. This is also a problem for a load of fix types that can’t be fixed by the authors because of the Versioning issue - so all Born Inactive content has to have back end fixes because they’ve already been versioned - this also gets very messy!
3. Also this is a problem for new changes to the International Release which can’t get include in the International Edition - like Yong’s SEP refsets!  Next time might be slightly easier because we may have HM changes to merge these Delta’s available (if not descoped), but still MUCH better to incorporate all authored content in the termServer or refset/Mapping tools…
4. Also a problem for the simpleMap files, as the CTV3 map comes out of the termServer, but the ICD-0 map comes from the mapping tool, so we need to manually merge the 2 into one simpleMap file. This means that if any impact o CTV3 from alpha/beta changes, we need to manually export the CTV3 changes from the termServer again, and re-merge the files!  So would be so much better to either host ICD-0 in the termServer, or both CTV3 and ICD-0 in the mapping tool!

1. Michael hoping to allow them to start doing a lot of these fixes themselves - for example allowing the release lead to delete born inactive content during the release cycle - AAT to raise infra ticket
2. Hunter work should also improve this situation
   
   

We need to fix the conflicts that can happen when runnig multiple jenkins jobs, or possibly even jenkins jobs vs validation runs!  

1. so if for example i’m running a full international build, but someone kicks of an sca validation run, will that clash? 
2. could this explain the 409 conflicts i sometimes get when trying to run the international builds, which then magically disappear and fix themselves before we find the root cause?
3. peter had a look but was not sure that the code for returning a sensible error when an export is already underway was done however, this ticket is still at "assigned" to b2i:   https://jira.ihtsdotools.org/browse/infra-1220
4. either way we need to be able to run multiple concurrent builds!!!

1. Hunter work to improve this situation - after hunter complete their work, we should update the status management of the release process as per Michael’s idea, to allow multiple contiguous builds

Need to discuss whether or not we should be going back to the uktc and telling them we will no longer consider any rf1 defects as part of the current release cycle - only as part of ongoing editing cycles for future releases?  (only issue here is that i no longer have a friendly contact in the uktc, or even someone who will talk to us or answer any of our emails!  so could be an issue getting the message across, as might sound harsh via email…)

1. Rory confirmed any RF1 issues that have their roots in RF2 still need to be fixed, but anything RF1 specific we will no longer fix - but no need to make noise about it now, just wait until the next set of RF1 only defects come through

Traceability database logging - is there any way to break down the logs for concepts which have huge logs? example was isrs-230, where we couldn’t track the root cause of the issue because the log for just one of the 3 concepts in question was over 11mb!!  the logging needs to cut off at 1mb otherwise it’ll break the indexes, so at the moment it just completely fails to log for any large complex changes!  if we could break it down into smaller chunks then this’d be great, as given the likelihood of content authoring continuing to be complex going forward (with all the remodelling work coming down the pipeline), we otherwise run the risk of having holes in our audit trail, making it more time-consuming to track issues down.

1. AAT to raise infra ticket to say even if a log is too large then truncate it before the limit to prevent crashing.

How realistic would it be to cut off the content slightly earlier than normal next release cycle?  As we saw from the craziness in this release, the volume and complexity of the authoring changes meant we were working long hours right up until the very last day of the release!  this is fine from my perspective, but it will impact the content release lead if this continues, plus it’s a huge risk to the release, as human error is far more likely to creep in if we’re working that late.